Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 898316 Details for
Bug 1100219
[RHSC] Node moved to Non-operational state when the cluster was imported to RHSC
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
engine logs
engine.log (text/x-log), 248.30 KB, created by
Shruti Sampat
on 2014-05-22 10:03:12 UTC
(
hide
)
Description:
engine logs
Filename:
MIME Type:
Creator:
Shruti Sampat
Created:
2014-05-22 10:03:12 UTC
Size:
248.30 KB
patch
obsolete
>2014-05-22 15:14:12,409 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-05-22 15:14:12,410 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-05-22 15:14:12,410 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhsc-doc.conf". >2014-05-22 15:14:12,411 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-05-22 15:14:12,411 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-05-22 15:14:12,412 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-05-22 15:14:12,412 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-05-22 15:14:12,412 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-05-22 15:14:12,413 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-access-plugin.conf". >2014-05-22 15:14:12,413 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-05-22 15:14:12,414 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_AJP_PORT" is "8702". >2014-05-22 15:14:12,414 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_APPS" is "engine.ear "/usr/share/ovirt-engine/branding/rhev-1.brand/applications/rhev.ear" redhat-access-plugin-rhsc.ear". >2014-05-22 15:14:12,414 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-05-22 15:14:12,414 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-05-22 15:14:12,414 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_HOST" is "localhost". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-05-22 15:14:12,415 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_PORT" is "5432". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_SECURED" is "False". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DB_USER" is "engine". >2014-05-22 15:14:12,416 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_FQDN" is "rhs-client19.lab.eng.blr.redhat.com". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_GROUP" is "ovirt". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-05-22 15:14:12,417 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_HTTP_PORT" is "None". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-05-22 15:14:12,418 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhsc-doc/manual". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PERM_MAX" is "256m". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PERM_MIN" is "256m". >2014-05-22 15:14:12,419 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-05-22 15:14:12,420 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-05-22 15:14:12,421 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_STOP_TIME" is "10". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-05-22 15:14:12,422 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_USER" is "ovirt". >2014-05-22 15:14:12,423 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-05-22 15:14:12,423 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-05-22 15:14:12,423 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-05-22 15:14:12,423 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-05-22 15:14:12,423 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 40) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-05-22 15:14:13,611 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Start initializing Backend >2014-05-22 15:14:13,908 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Running ovirt-engine 3.0.0-0.5.master.el6_5 >2014-05-22 15:14:13,910 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 41) Start initializing dictionaries >2014-05-22 15:14:13,914 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 41) Finished initializing dictionaries >2014-05-22 15:14:13,915 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Start initializing AuditLogCleanupManager >2014-05-22 15:14:13,916 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-05-22 15:14:13,932 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Finished initializing AuditLogCleanupManager >2014-05-22 15:14:13,934 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Start initializing TagsDirector >2014-05-22 15:14:13,942 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Tag root added to tree >2014-05-22 15:14:14,005 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Finished initializing TagsDirector >2014-05-22 15:14:14,008 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 41) Start initializing IsoDomainListSyncronizer >2014-05-22 15:14:14,014 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 41) Finished initializing IsoDomainListSyncronizer >2014-05-22 15:14:14,025 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 41) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-05-22 15:14:14,277 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Completed initializing handlers >2014-05-22 15:14:14,278 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Start initializing ErrorTranslatorImpl >2014-05-22 15:14:14,288 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-05-22 15:14:14,288 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Finished initializing ErrorTranslatorImpl >2014-05-22 15:14:14,289 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Start initializing ErrorTranslatorImpl >2014-05-22 15:14:14,289 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Finished initializing ErrorTranslatorImpl >2014-05-22 15:14:14,289 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Mark incomplete jobs as UNKNOWN >2014-05-22 15:14:14,386 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 41) Start initializing JobRepositoryCleanupManager >2014-05-22 15:14:14,387 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 41) Finished initializing JobRepositoryCleanupManager >2014-05-22 15:14:14,389 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 41) Start initializing AutoRecoveryManager >2014-05-22 15:14:14,391 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 41) Finished initializing AutoRecoveryManager >2014-05-22 15:14:14,393 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 41) Start initializing ExecutionMessageDirector >2014-05-22 15:14:14,394 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 41) Finished initializing ExecutionMessageDirector >2014-05-22 15:14:14,440 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 44) Start initializing UsersDomainsCacheManagerService >2014-05-22 15:14:14,447 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 44) Start initializing DbUserCacheManager >2014-05-22 15:14:14,447 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 44) Finished initializing DbUserCacheManager >2014-05-22 15:14:14,447 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 44) Finished initializing UsersDomainsCacheManagerService >2014-05-22 15:14:14,459 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ServerService Thread Pool -- 41) Initialization of AsyncTaskManager completed successfully. >2014-05-22 15:14:14,464 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 41) Start initializing ResourceManager >2014-05-22 15:14:14,599 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 41) Finished initializing ResourceManager >2014-05-22 15:14:14,606 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 41) Initialization of OvfDataUpdater completed successfully. >2014-05-22 15:14:14,609 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Start scheduling to enable vds load balancer >2014-05-22 15:14:14,610 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Finished scheduling to enable vds load balancer >2014-05-22 15:14:14,610 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Start HA Reservation check >2014-05-22 15:14:14,610 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Finished HA Reservation check >2014-05-22 15:14:14,625 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (org.ovirt.thread.pool-4-thread-1) MacPoolManager(311dc34): Start initializing >2014-05-22 15:14:14,638 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 41) Init VM custom properties utilities >2014-05-22 15:14:14,650 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (org.ovirt.thread.pool-4-thread-1) MacPoolManager(311dc34): Finished initializing. Available MACs in pool: 256 >2014-05-22 15:14:14,673 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 41) Init device custom properties utilities >2014-05-22 15:14:14,677 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Initializing Scheduling manager >2014-05-22 15:14:14,732 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) External scheduler disabled, discovery skipped >2014-05-22 15:14:14,732 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 41) Initialized Scheduling manager >2014-05-22 15:14:14,733 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 41) Initializing DWH Heart Beat >2014-05-22 15:14:14,734 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 41) DWH Heart Beat initialized >2014-05-22 15:15:14,680 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2014-05-22 15:15:14,694 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:16:14,700 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-4) No up server in cluster >2014-05-22 15:16:14,700 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-4) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:17:14,705 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2014-05-22 15:17:14,705 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:17:55,949 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-5) Theme /80-rhs-1.brand has no property defined for key resources >2014-05-22 15:18:07,413 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/gluster-nagios-monitoring.json] >2014-05-22 15:18:07,424 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/gluster-nagios-monitoring-config.json] >2014-05-22 15:18:07,425 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc.json] >2014-05-22 15:18:07,426 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-2) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc-config.json] >2014-05-22 15:18:07,920 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-4) Theme /80-rhs-1.brand has no property defined for key resources >2014-05-22 15:18:12,637 INFO [org.ovirt.engine.core.bll.LoginBaseCommand] (ajp-/127.0.0.1:8702-3) Cant login user "admin" with authentication profile "internal" because the authentication failed. >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: UNASSIGNED not exist in string table >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-05-22 15:18:12,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: DWH_STOPPED not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: DWH_STARTED not exist in string table >2014-05-22 15:18:12,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: DWH_ERROR not exist in string table >2014-05-22 15:18:12,663 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2014-05-22 15:18:12,663 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2014-05-22 15:18:12,663 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-05-22 15:18:12,663 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2014-05-22 15:18:12,664 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_ADDED_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_TO_ADD_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_UPDATED_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_TO_UPDATE_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_REMOVED_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,665 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) AuditLogType: USER_FAILED_TO_REMOVE_AFFINITY_GROUP not have severity. Assumed Normal >2014-05-22 15:18:12,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin failed to log in. >2014-05-22 15:18:12,711 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-3) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-05-22 15:18:14,595 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginAdminUserCommand internal: false. >2014-05-22 15:18:14,709 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-9) No up server in cluster >2014-05-22 15:18:14,709 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-9) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:18:14,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-05-22 15:18:15,403 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-2) Can't read file "/var/lib/ovirt-engine/reports.xml" for request "/ovirt-engine/services/reports-ui", will send a 404 error response. >2014-05-22 15:18:16,759 ERROR [org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl] (ajp-/127.0.0.1:8702-1) Retrieving non string value from session >2014-05-22 15:18:58,194 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-7) [93e1bc6] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-000000000173 Type: StoragePool >2014-05-22 15:18:58,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [93e1bc6] Correlation ID: 93e1bc6, Call Stack: null, Custom Event ID: -1, Message: Cluster test was added by admin >2014-05-22 15:18:58,383 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-4-thread-3) [1240927] Start running CanDoAction for command number 1/2 (Command type: AddVds) >2014-05-22 15:18:58,383 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-4-thread-4) [2f2e9645] Start running CanDoAction for command number 2/2 (Command type: AddVds) >2014-05-22 15:18:58,934 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (org.ovirt.thread.pool-4-thread-4) [2f2e9645] Failed to initiate vdsm-id request on host with message Command returned failure code 1 during SSH session 'root@10.70.37.217' >2014-05-22 15:18:58,939 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-4-thread-4) [2f2e9645] Finish handling CanDoAction for command number 2/2 (Command type: AddVds) >2014-05-22 15:18:58,956 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (org.ovirt.thread.pool-4-thread-3) [1240927] Failed to initiate vdsm-id request on host with message Command returned failure code 1 during SSH session 'root@10.70.37.220' >2014-05-22 15:18:58,965 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-4-thread-3) [1240927] Finish handling CanDoAction for command number 1/2 (Command type: AddVds) >2014-05-22 15:18:59,059 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (org.ovirt.thread.pool-4-thread-5) [1240927] Running command: AddVdsCommand internal: false. Entities affected : ID: e4472500-c757-42cb-83b6-2ec63c4aa68a Type: VdsGroups >2014-05-22 15:18:59,165 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:18:59,193 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-000000000173 value: REGISTER_VDS >, sharedLocks= ] >2014-05-22 15:18:59,218 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] START, RemoveVdsVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 47529932 >2014-05-22 15:18:59,218 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] FINISH, RemoveVdsVDSCommand, log id: 47529932 >2014-05-22 15:18:59,221 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] START, AddVdsVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 7da01d1d >2014-05-22 15:18:59,221 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] AddVds - entered , starting logic to add VDS 52301919-41f3-4b25-9ebe-642d614ed5dd >2014-05-22 15:18:59,223 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] AddVds - VDS 52301919-41f3-4b25-9ebe-642d614ed5dd was added, will try to add it to the resource manager >2014-05-22 15:18:59,229 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-4-thread-5) [58d74986] Entered VdsManager constructor >2014-05-22 15:18:59,247 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-4-thread-5) [58d74986] Initialize vdsBroker (10.70.37.220,54,321) >2014-05-22 15:18:59,376 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (org.ovirt.thread.pool-4-thread-5) [58d74986] VDS 52301919-41f3-4b25-9ebe-642d614ed5dd was added to the Resource Manager >2014-05-22 15:18:59,376 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [58d74986] FINISH, AddVdsVDSCommand, log id: 7da01d1d >2014-05-22 15:18:59,422 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:18:59,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-5) [58d74986] Correlation ID: 1240927, Job ID: 7a39459b-3a82-4eaf-863c-ec7e3cfa2a07, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 was added by admin. >2014-05-22 15:18:59,442 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Running command: InstallVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:18:59,472 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Before Installation host 52301919-41f3-4b25-9ebe-642d614ed5dd, 10.70.37.220 >2014-05-22 15:18:59,475 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (org.ovirt.thread.pool-4-thread-5) [2f2e9645] Running command: AddVdsCommand internal: false. Entities affected : ID: e4472500-c757-42cb-83b6-2ec63c4aa68a Type: VdsGroups >2014-05-22 15:18:59,481 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 24168c47 >2014-05-22 15:18:59,504 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:18:59,509 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] FINISH, SetVdsStatusVDSCommand, log id: 24168c47 >2014-05-22 15:18:59,517 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-000000000173 value: REGISTER_VDS >, sharedLocks= ] >2014-05-22 15:18:59,521 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] START, RemoveVdsVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 140e1b6c >2014-05-22 15:18:59,521 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] FINISH, RemoveVdsVDSCommand, log id: 140e1b6c >2014-05-22 15:18:59,523 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] START, AddVdsVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 3f1f6990 >2014-05-22 15:18:59,523 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] AddVds - entered , starting logic to add VDS 5f5c1a08-edd9-4de8-847e-701131ec8d30 >2014-05-22 15:18:59,524 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] AddVds - VDS 5f5c1a08-edd9-4de8-847e-701131ec8d30 was added, will try to add it to the resource manager >2014-05-22 15:18:59,525 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] Entered VdsManager constructor >2014-05-22 15:18:59,542 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] Initialize vdsBroker (10.70.37.217,54,321) >2014-05-22 15:18:59,543 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] VDS 5f5c1a08-edd9-4de8-847e-701131ec8d30 was added to the Resource Manager >2014-05-22 15:18:59,543 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] FINISH, AddVdsVDSCommand, log id: 3f1f6990 >2014-05-22 15:18:59,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-5) [51d4bb74] Correlation ID: 2f2e9645, Job ID: db160b13-eb70-4439-b18e-b9c1e2789728, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 was added by admin. >2014-05-22 15:18:59,585 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:18:59,586 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Installation 10.70.37.220: Connected to host 10.70.37.220 with SSH key fingerprint: ed:dc:a1:07:8f:32:c3:58:5e:4d:ac:84:bd:77:b5:78 >2014-05-22 15:18:59,609 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Connected to host 10.70.37.220 with SSH key fingerprint: ed:dc:a1:07:8f:32:c3:58:5e:4d:ac:84:bd:77:b5:78. >2014-05-22 15:18:59,617 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] Running command: InstallVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:18:59,617 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] Before Installation host 5f5c1a08-edd9-4de8-847e-701131ec8d30, 10.70.37.217 >2014-05-22 15:18:59,622 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 64a014a7 >2014-05-22 15:18:59,634 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] FINISH, SetVdsStatusVDSCommand, log id: 64a014a7 >2014-05-22 15:18:59,637 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Installation of 10.70.37.220. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-05-22 15:18:59,638 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (org.ovirt.thread.pool-4-thread-6) Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' is missing, creating >2014-05-22 15:18:59,704 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (org.ovirt.thread.pool-4-thread-6) SSH execute root@10.70.37.220 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-05-22 15:18:59,716 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-4-thread-7) [30f68097] Installation 10.70.37.217: Connected to host 10.70.37.217 with SSH key fingerprint: 78:91:29:48:2b:17:4d:f7:27:14:6a:60:a7:78:f5:27 >2014-05-22 15:18:59,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-7) [30f68097] Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Connected to host 10.70.37.217 with SSH key fingerprint: 78:91:29:48:2b:17:4d:f7:27:14:6a:60:a7:78:f5:27. >2014-05-22 15:18:59,769 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-4-thread-7) [30f68097] Installation of 10.70.37.217. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-05-22 15:18:59,770 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (org.ovirt.thread.pool-4-thread-7) SSH execute root@10.70.37.217 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-05-22 15:19:00,247 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Initializing >2014-05-22 15:19:00,262 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Initializing >2014-05-22 15:19:00,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Initializing. >2014-05-22 15:19:00,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Initializing. >2014-05-22 15:19:00,300 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Environment setup >2014-05-22 15:19:00,305 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Environment setup >2014-05-22 15:19:00,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Environment setup. >2014-05-22 15:19:00,319 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Environment packages setup >2014-05-22 15:19:00,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Environment setup. >2014-05-22 15:19:00,325 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Environment packages setup >2014-05-22 15:19:00,334 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Environment packages setup. >2014-05-22 15:19:00,342 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Environment packages setup. >2014-05-22 15:19:00,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Programs detection >2014-05-22 15:19:00,691 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Programs detection >2014-05-22 15:19:00,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Programs detection. >2014-05-22 15:19:00,709 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Environment customization >2014-05-22 15:19:00,717 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Programs detection. >2014-05-22 15:19:00,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Environment customization >2014-05-22 15:19:00,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Environment customization. >2014-05-22 15:19:00,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Environment customization. >2014-05-22 15:19:00,744 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Logs at host located at: '/tmp/ovirt-host-deploy-20140522105824.log' >2014-05-22 15:19:00,749 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Logs at host located at: '/tmp/ovirt-host-deploy-20140522105824.log' >2014-05-22 15:19:00,758 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Logs at host located at: '/tmp/ovirt-host-deploy-20140522105824.log'. >2014-05-22 15:19:00,767 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Logs at host located at: '/tmp/ovirt-host-deploy-20140522105824.log'. >2014-05-22 15:19:00,839 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host 10.70.37.217 reports unique id 7AC5CED6-15AF-91E3-E895-7DCF8F1E3C51 >2014-05-22 15:19:00,841 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host 10.70.37.220 reports unique id D723BACB-0A96-8122-BE77-B160F812C14F >2014-05-22 15:19:00,864 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 7AC5CED6-15AF-91E3-E895-7DCF8F1E3C51 to Host 10.70.37.217 >2014-05-22 15:19:00,864 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id D723BACB-0A96-8122-BE77-B160F812C14F to Host 10.70.37.220 >2014-05-22 15:19:01,037 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Setup validation >2014-05-22 15:19:01,049 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Setup validation >2014-05-22 15:19:01,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Setup validation. >2014-05-22 15:19:01,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Setup validation. >2014-05-22 15:19:02,381 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Initializing Host: 10.70.37.220 >2014-05-22 15:19:02,510 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Cannot validate host name settings, reason: cannot resolve own name 'rhs.denali.1' >2014-05-22 15:19:02,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs.denali.1'. >2014-05-22 15:19:02,545 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Initializing Host: 10.70.37.217 >2014-05-22 15:19:02,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Transaction setup >2014-05-22 15:19:02,708 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Transaction setup. >2014-05-22 15:19:02,795 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Misc configuration >2014-05-22 15:19:02,808 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Misc configuration. >2014-05-22 15:19:02,808 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Package installation >2014-05-22 15:19:02,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Package installation. >2014-05-22 15:19:03,637 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Cannot validate host name settings, reason: cannot resolve own name 'rhs.denali.2' >2014-05-22 15:19:03,649 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs.denali.2'. >2014-05-22 15:19:03,775 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Misc configuration >2014-05-22 15:19:03,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Misc configuration. >2014-05-22 15:19:03,828 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Transaction setup >2014-05-22 15:19:03,841 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Transaction setup. >2014-05-22 15:19:03,948 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Misc configuration >2014-05-22 15:19:03,955 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Setting time >2014-05-22 15:19:03,957 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Misc configuration. >2014-05-22 15:19:03,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Package installation >2014-05-22 15:19:03,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Setting time. >2014-05-22 15:19:03,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Package installation. >2014-05-22 15:19:04,980 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Misc configuration >2014-05-22 15:19:05,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Misc configuration. >2014-05-22 15:19:05,074 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Setting time >2014-05-22 15:19:05,114 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Setting time. >2014-05-22 15:19:06,421 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Enrolling certificate >2014-05-22 15:19:06,448 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Enrolling certificate. >2014-05-22 15:19:07,513 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Enrolling certificate >2014-05-22 15:19:07,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Enrolling certificate. >2014-05-22 15:19:07,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Transaction commit >2014-05-22 15:19:07,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Transaction commit. >2014-05-22 15:19:07,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Closing up >2014-05-22 15:19:07,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Closing up. >2014-05-22 15:19:08,296 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Starting gluster >2014-05-22 15:19:08,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Starting gluster. >2014-05-22 15:19:09,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Transaction commit >2014-05-22 15:19:09,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Transaction commit. >2014-05-22 15:19:09,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Closing up >2014-05-22 15:19:09,297 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Closing up. >2014-05-22 15:19:09,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Starting gluster >2014-05-22 15:19:09,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Starting gluster. >2014-05-22 15:19:14,712 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-24) No up server in cluster >2014-05-22 15:19:14,713 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-24) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:19:14,715 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-24) No up server in cluster >2014-05-22 15:19:14,715 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-24) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:19:18,261 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stopping libvirtd >2014-05-22 15:19:18,296 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stopping libvirtd. >2014-05-22 15:19:18,517 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stopping libvirtd >2014-05-22 15:19:18,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stopping libvirtd. >2014-05-22 15:19:18,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Starting vdsm >2014-05-22 15:19:18,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Starting vdsm. >2014-05-22 15:19:19,000 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Starting vdsm >2014-05-22 15:19:19,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Starting vdsm. >2014-05-22 15:19:22,321 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Pre-termination >2014-05-22 15:19:22,353 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Pre-termination. >2014-05-22 15:19:22,366 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140522151922-10.70.37.217-30f68097.log' >2014-05-22 15:19:22,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140522151922-10.70.37.217-30f68097.log'. >2014-05-22 15:19:22,450 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Pre-termination >2014-05-22 15:19:22,461 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Pre-termination. >2014-05-22 15:19:22,482 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140522151922-10.70.37.220-4c786c6e.log' >2014-05-22 15:19:22,495 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140522151922-10.70.37.220-4c786c6e.log'. >2014-05-22 15:19:22,626 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.217: Stage: Termination >2014-05-22 15:19:22,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 30f68097, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.217. Stage: Termination. >2014-05-22 15:19:22,724 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.70.37.220: Stage: Termination >2014-05-22 15:19:22,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4c786c6e, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.70.37.220. Stage: Termination. >2014-05-22 15:19:22,878 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:22,891 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] Command PollVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:22,922 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:22,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Command PollVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:23,397 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:23,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-7) [30f68097] Command PollVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:23,430 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:23,432 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Command PollVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException >2014-05-22 15:19:24,089 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-4-thread-6) [4c786c6e] Engine managed to communicate with VDSM agent on host 10.70.37.220 >2014-05-22 15:19:24,089 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-4-thread-7) [30f68097] Engine managed to communicate with VDSM agent on host 10.70.37.217 >2014-05-22 15:19:24,426 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:19:24,431 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] START, SetupNetworksVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-000000000173, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=false, required=true, migration=false}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth0 {id=a2cc88ed-4a24-4969-9c95-504cdbd5d848, vdsId=5f5c1a08-edd9-4de8-847e-701131ec8d30, macAddress=52:54:00:5c:15:27, networkName=ovirtmgmt, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.70.37.217, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null}], > removedNetworks=[], > removedBonds=[]), log id: 3bebe4c3 >2014-05-22 15:19:24,433 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:19:24,435 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Failed in SetupNetworksVDS method, for vds: 10.70.37.217; host: 10.70.37.217 >2014-05-22 15:19:24,435 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Command SetupNetworksVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-000000000173, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=false, required=true, migration=false}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth0 {id=a2cc88ed-4a24-4969-9c95-504cdbd5d848, vdsId=5f5c1a08-edd9-4de8-847e-701131ec8d30, macAddress=52:54:00:5c:15:27, networkName=ovirtmgmt, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.70.37.217, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null}], > removedNetworks=[], > removedBonds=[]) execution failed. Exception: NoSuchElementException: >2014-05-22 15:19:24,435 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] START, SetupNetworksVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-000000000173, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=false, required=true, migration=false}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth0 {id=e8d47cd3-6dfc-4cbc-b842-4d88ddf3ef99, vdsId=52301919-41f3-4b25-9ebe-642d614ed5dd, macAddress=52:54:00:de:d4:c3, networkName=ovirtmgmt, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.70.37.220, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null}], > removedNetworks=[], > removedBonds=[]), log id: 6c2d9f1e >2014-05-22 15:19:24,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] FINISH, SetupNetworksVDSCommand, log id: 3bebe4c3 >2014-05-22 15:19:24,438 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Failed in SetupNetworksVDS method, for vds: 10.70.37.220; host: 10.70.37.220 >2014-05-22 15:19:24,438 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Command SetupNetworksVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-000000000173, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=false, required=true, migration=false}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth0 {id=e8d47cd3-6dfc-4cbc-b842-4d88ddf3ef99, vdsId=52301919-41f3-4b25-9ebe-642d614ed5dd, macAddress=52:54:00:de:d4:c3, networkName=ovirtmgmt, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.70.37.220, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null}], > removedNetworks=[], > removedBonds=[]) execution failed. Exception: NoSuchElementException: >2014-05-22 15:19:24,438 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] FINISH, SetupNetworksVDSCommand, log id: 6c2d9f1e >2014-05-22 15:19:24,436 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw exception: java.lang.NullPointerException > at org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.isDone(FutureVDSCommand.java:41) [vdsbroker.jar:] > at org.ovirt.engine.core.bll.network.host.SetupNetworksCommand.pollInterruptively(SetupNetworksCommand.java:212) [bll.jar:] > at org.ovirt.engine.core.bll.network.host.SetupNetworksCommand.executeCommand(SetupNetworksCommand.java:133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:341) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:226) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:69) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:341) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:254) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:634) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:163) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:97) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_55] > >2014-05-22 15:19:24,438 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw exception: java.lang.NullPointerException > at org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.isDone(FutureVDSCommand.java:41) [vdsbroker.jar:] > at org.ovirt.engine.core.bll.network.host.SetupNetworksCommand.pollInterruptively(SetupNetworksCommand.java:212) [bll.jar:] > at org.ovirt.engine.core.bll.network.host.SetupNetworksCommand.executeCommand(SetupNetworksCommand.java:133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:341) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:226) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:69) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:341) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:254) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:634) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:163) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:97) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_55] > >2014-05-22 15:19:24,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-6) [684b2556] Failed to configure management network: Failed to configure management network on host 10.70.37.220 due to setup networks failure. >2014-05-22 15:19:24,461 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Host installation failed for host 52301919-41f3-4b25-9ebe-642d614ed5dd, 10.70.37.220.: org.ovirt.engine.core.bll.InstallVdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:349) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:254) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:634) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:163) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:97) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_55] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: Failed to configure management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:241) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:69) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:341) [bll.jar:] > ... 61 more > >2014-05-22 15:19:24,466 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1f9ee05d >2014-05-22 15:19:24,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Failed to configure management network: Failed to configure management network on host 10.70.37.217 due to setup networks failure. >2014-05-22 15:19:24,469 ERROR [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Host installation failed for host 5f5c1a08-edd9-4de8-847e-701131ec8d30, 10.70.37.217.: org.ovirt.engine.core.bll.InstallVdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:349) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.installHost(InstallVdsCommand.java:254) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:392) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:634) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.3.2.Final-redhat-2] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:163) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:97) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_55] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: Failed to configure management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:241) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:69) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsCommand.configureManagementNetwork(InstallVdsCommand.java:341) [bll.jar:] > ... 61 more > >2014-05-22 15:19:24,474 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 14b8ccad >2014-05-22 15:19:24,477 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] FINISH, SetVdsStatusVDSCommand, log id: 1f9ee05d >2014-05-22 15:19:24,493 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] FINISH, SetVdsStatusVDSCommand, log id: 14b8ccad >2014-05-22 15:19:24,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-6) [684b2556] Correlation ID: 4c786c6e, Job ID: 7a39459b-3a82-4eaf-863c-ec7e3cfa2a07, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 installation failed. Failed to configure management network on the host. >2014-05-22 15:19:24,510 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Correlation ID: 30f68097, Job ID: db160b13-eb70-4439-b18e-b9c1e2789728, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 installation failed. Failed to configure management network on the host. >2014-05-22 15:19:24,519 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-7) [5ebbd304] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:19:24,527 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (org.ovirt.thread.pool-4-thread-6) [684b2556] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:19:26,524 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 65c016d3 >2014-05-22 15:19:26,643 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GetHardwareInfoVDSCommand, log id: 65c016d3 >2014-05-22 15:19:26,668 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: b58b179 >2014-05-22 15:19:26,751 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:19:26,753 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: beb77e3 >2014-05-22 15:19:26,759 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] FINISH, SetVdsStatusVDSCommand, log id: beb77e3 >2014-05-22 15:19:26,779 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GetHardwareInfoVDSCommand, log id: b58b179 >2014-05-22 15:19:26,795 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:19:26,826 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] Correlation ID: 19d1ec0e, Job ID: 55cfecdf-0d1e-4e7d-be79-ea864838186c, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:19:26,862 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: dfdef7c >2014-05-22 15:19:26,868 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:19:26,871 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 3aca9f55 >2014-05-22 15:19:26,884 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] FINISH, SetVdsStatusVDSCommand, log id: 3aca9f55 >2014-05-22 15:19:26,889 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:19:26,901 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] Correlation ID: 1da9e9e3, Job ID: 306c44b5-804b-4107-9121-b33e60681333, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:19:26,919 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: b243322 >2014-05-22 15:19:27,216 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: dfdef7c >2014-05-22 15:19:27,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) [19d1ec0e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:19:27,272 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: b243322 >2014-05-22 15:19:27,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) [1da9e9e3] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:19:27,368 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-51) [42da35c3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:19:27,388 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-53) [467d7e19] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:20:00,046 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-29) Autorecovering 2 hosts >2014-05-22 15:20:00,047 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-29) Autorecovering hosts id: 52301919-41f3-4b25-9ebe-642d614ed5dd, name : 10.70.37.220 >2014-05-22 15:20:00,054 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:20:00,055 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:20:00,056 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Before acquiring lock in order to prevent monitoring for host 10.70.37.220 from data-center Default >2014-05-22 15:20:00,056 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.220 from data-center Default >2014-05-22 15:20:00,068 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3cb7035a >2014-05-22 15:20:00,098 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] FINISH, SetVdsStatusVDSCommand, log id: 3cb7035a >2014-05-22 15:20:00,123 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Activate finished. Lock released. Monitoring can run now for host 10.70.37.220 from data-center Default >2014-05-22 15:20:00,139 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [19210d28] Correlation ID: 19210d28, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 was autorecovered. >2014-05-22 15:20:00,140 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [19210d28] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:20:00,140 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-29) [19210d28] Autorecovering hosts id: 5f5c1a08-edd9-4de8-847e-701131ec8d30, name : 10.70.37.217 >2014-05-22 15:20:00,144 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:20:00,144 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:20:00,144 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Before acquiring lock in order to prevent monitoring for host 10.70.37.217 from data-center Default >2014-05-22 15:20:00,145 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.217 from data-center Default >2014-05-22 15:20:00,150 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 424b5f17 >2014-05-22 15:20:00,164 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] FINISH, SetVdsStatusVDSCommand, log id: 424b5f17 >2014-05-22 15:20:00,181 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Activate finished. Lock released. Monitoring can run now for host 10.70.37.217 from data-center Default >2014-05-22 15:20:00,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [5a229d93] Correlation ID: 5a229d93, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 was autorecovered. >2014-05-22 15:20:00,189 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-29) [5a229d93] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:20:00,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 7cfca360 >2014-05-22 15:20:00,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: 98a51bb >2014-05-22 15:20:00,996 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GetHardwareInfoVDSCommand, log id: 7cfca360 >2014-05-22 15:20:00,998 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GetHardwareInfoVDSCommand, log id: 98a51bb >2014-05-22 15:20:01,097 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:20:01,101 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: bc45581 >2014-05-22 15:20:01,106 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:20:01,108 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 1df09919 >2014-05-22 15:20:01,114 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] FINISH, SetVdsStatusVDSCommand, log id: bc45581 >2014-05-22 15:20:01,120 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:20:01,122 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] FINISH, SetVdsStatusVDSCommand, log id: 1df09919 >2014-05-22 15:20:01,127 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:20:01,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 4f499237 >2014-05-22 15:20:01,157 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 5dfbc49d >2014-05-22 15:20:01,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) [576491bd] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: 4f499237 >2014-05-22 15:20:01,489 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) [576491bd] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:20:01,497 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) [25d2f393] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: 5dfbc49d >2014-05-22 15:20:01,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [25d2f393] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:20:01,554 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-32) [76f4b528] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:20:01,555 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) [76f4b528] Host 52301919-41f3-4b25-9ebe-642d614ed5dd : 10.70.37.220 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:20:01,577 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-33) [3addb7bb] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:20:01,579 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) [3addb7bb] Host 5f5c1a08-edd9-4de8-847e-701131ec8d30 : 10.70.37.217 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:20:14,718 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-65) No up server in cluster >2014-05-22 15:20:14,719 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-65) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:20:14,721 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-65) No up server in cluster >2014-05-22 15:20:14,721 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-65) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:20:35,004 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-34) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:20:50,218 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-46) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:21:14,722 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-16) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:21:14,724 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2014-05-22 15:21:14,724 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:21:14,726 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2014-05-22 15:21:14,726 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:21:17,739 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-18) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:22:14,731 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-46) No up server in cluster >2014-05-22 15:22:14,732 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-46) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:22:14,734 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-46) No up server in cluster >2014-05-22 15:22:14,734 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-46) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:22:18,547 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-16) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:22:30,730 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-26) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:22:33,749 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-28) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:23:14,738 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-88) No up server in cluster >2014-05-22 15:23:14,738 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-88) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:23:14,741 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-88) No up server in cluster >2014-05-22 15:23:14,741 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-88) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:23:37,627 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-28) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:24:14,749 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-35) No up server in cluster >2014-05-22 15:24:14,749 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-35) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:24:14,752 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-35) No up server in cluster >2014-05-22 15:24:14,752 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-35) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:24:53,775 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-40) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:25:00,046 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-40) Autorecovering 2 hosts >2014-05-22 15:25:00,047 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-40) Autorecovering hosts id: 52301919-41f3-4b25-9ebe-642d614ed5dd, name : 10.70.37.220 >2014-05-22 15:25:00,050 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:25:00,051 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:25:00,051 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Before acquiring lock in order to prevent monitoring for host 10.70.37.220 from data-center Default >2014-05-22 15:25:00,051 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.220 from data-center Default >2014-05-22 15:25:00,057 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 40676da2 >2014-05-22 15:25:00,081 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] FINISH, SetVdsStatusVDSCommand, log id: 40676da2 >2014-05-22 15:25:00,098 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Activate finished. Lock released. Monitoring can run now for host 10.70.37.220 from data-center Default >2014-05-22 15:25:00,100 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [537a1736] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:25:00,101 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-40) [537a1736] Autorecovering hosts id: 5f5c1a08-edd9-4de8-847e-701131ec8d30, name : 10.70.37.217 >2014-05-22 15:25:00,103 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:25:00,103 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:25:00,103 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Before acquiring lock in order to prevent monitoring for host 10.70.37.217 from data-center Default >2014-05-22 15:25:00,104 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.217 from data-center Default >2014-05-22 15:25:00,110 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 782e458a >2014-05-22 15:25:00,123 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] FINISH, SetVdsStatusVDSCommand, log id: 782e458a >2014-05-22 15:25:00,131 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Activate finished. Lock released. Monitoring can run now for host 10.70.37.217 from data-center Default >2014-05-22 15:25:00,133 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-40) [32be2b58] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:25:02,983 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 3ef335f0 >2014-05-22 15:25:02,987 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: 2b70a971 >2014-05-22 15:25:03,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GetHardwareInfoVDSCommand, log id: 3ef335f0 >2014-05-22 15:25:03,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GetHardwareInfoVDSCommand, log id: 2b70a971 >2014-05-22 15:25:03,147 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:25:03,150 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 5e626cc8 >2014-05-22 15:25:03,156 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:25:03,158 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 6da64a91 >2014-05-22 15:25:03,164 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] FINISH, SetVdsStatusVDSCommand, log id: 5e626cc8 >2014-05-22 15:25:03,169 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:25:03,173 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] FINISH, SetVdsStatusVDSCommand, log id: 6da64a91 >2014-05-22 15:25:03,180 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:25:03,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [6c787d5] Correlation ID: 6c787d5, Job ID: 4ff99af5-7f36-4563-a8df-49f8adabe22c, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:25:03,197 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) [220ec562] Correlation ID: 220ec562, Job ID: 2e2690b1-ca6c-4f5f-8713-79b35560afbc, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:25:03,224 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 666a35d0 >2014-05-22 15:25:03,267 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 31c737b3 >2014-05-22 15:25:03,567 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) [6c787d5] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: 666a35d0 >2014-05-22 15:25:03,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [6c787d5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:25:03,615 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) [220ec562] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: 31c737b3 >2014-05-22 15:25:03,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) [220ec562] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:25:03,643 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-47) [2dcd8834] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:25:03,645 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) [2dcd8834] Host 5f5c1a08-edd9-4de8-847e-701131ec8d30 : 10.70.37.217 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:25:03,685 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-48) [20ebbaa] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:25:03,686 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) [20ebbaa] Host 52301919-41f3-4b25-9ebe-642d614ed5dd : 10.70.37.220 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:25:14,756 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-74) No up server in cluster >2014-05-22 15:25:14,756 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-74) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:25:14,759 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-74) No up server in cluster >2014-05-22 15:25:14,759 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-74) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:26:07,446 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-5) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:26:10,495 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-10) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:26:14,763 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-18) No up server in cluster >2014-05-22 15:26:14,763 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-18) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:26:14,765 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-18) No up server in cluster >2014-05-22 15:26:14,765 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-18) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:26:19,641 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-7) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:26:22,661 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-17) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:26:25,679 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-18) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:27:14,769 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-59) No up server in cluster >2014-05-22 15:27:14,770 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-59) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:27:14,771 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-59) No up server in cluster >2014-05-22 15:27:14,772 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-59) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:28:14,776 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-99) No up server in cluster >2014-05-22 15:28:14,777 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-99) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:28:14,779 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-99) No up server in cluster >2014-05-22 15:28:14,779 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-99) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:28:39,446 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-24) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:29:05,188 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-2) Theme /80-rhs-1.brand has no property defined for key resources >2014-05-22 15:29:11,930 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-4) Theme /80-rhs-1.brand has no property defined for key resources >2014-05-22 15:29:14,782 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-47) [2dcd8834] No up server in cluster >2014-05-22 15:29:14,783 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-47) [2dcd8834] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:29:14,784 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-47) [2dcd8834] No up server in cluster >2014-05-22 15:29:14,784 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-47) [2dcd8834] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:29:30,098 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginAdminUserCommand internal: false. >2014-05-22 15:29:30,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-05-22 15:29:30,199 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-05-22 15:29:30,349 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-8) Can't read file "/var/lib/ovirt-engine/reports.xml" for request "/ovirt-engine/services/reports-ui", will send a 404 error response. >2014-05-22 15:29:37,207 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-23) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:29:40,223 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-25) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:29:58,427 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-38) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:30:00,050 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-54) Autorecovering 2 hosts >2014-05-22 15:30:00,050 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-54) Autorecovering hosts id: 52301919-41f3-4b25-9ebe-642d614ed5dd, name : 10.70.37.220 >2014-05-22 15:30:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:30:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:30:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Before acquiring lock in order to prevent monitoring for host 10.70.37.220 from data-center Default >2014-05-22 15:30:00,054 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.220 from data-center Default >2014-05-22 15:30:00,060 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6734f58b >2014-05-22 15:30:00,073 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] FINISH, SetVdsStatusVDSCommand, log id: 6734f58b >2014-05-22 15:30:00,081 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Activate finished. Lock released. Monitoring can run now for host 10.70.37.220 from data-center Default >2014-05-22 15:30:00,083 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [5aae8995] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:30:00,083 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-54) [5aae8995] Autorecovering hosts id: 5f5c1a08-edd9-4de8-847e-701131ec8d30, name : 10.70.37.217 >2014-05-22 15:30:00,086 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:30:00,086 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:30:00,086 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Before acquiring lock in order to prevent monitoring for host 10.70.37.217 from data-center Default >2014-05-22 15:30:00,086 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.217 from data-center Default >2014-05-22 15:30:00,093 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4eee4dff >2014-05-22 15:30:00,106 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] FINISH, SetVdsStatusVDSCommand, log id: 4eee4dff >2014-05-22 15:30:00,123 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Activate finished. Lock released. Monitoring can run now for host 10.70.37.217 from data-center Default >2014-05-22 15:30:00,125 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-54) [1363183d] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:30:01,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: 3f9bbb7f >2014-05-22 15:30:01,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 5e9e35a6 >2014-05-22 15:30:01,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GetHardwareInfoVDSCommand, log id: 3f9bbb7f >2014-05-22 15:30:01,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GetHardwareInfoVDSCommand, log id: 5e9e35a6 >2014-05-22 15:30:01,756 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:30:01,759 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 4e83cfb5 >2014-05-22 15:30:01,764 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:30:01,767 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 293b7411 >2014-05-22 15:30:01,772 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] FINISH, SetVdsStatusVDSCommand, log id: 4e83cfb5 >2014-05-22 15:30:01,778 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:30:01,781 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] FINISH, SetVdsStatusVDSCommand, log id: 293b7411 >2014-05-22 15:30:01,787 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:30:01,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-58) [23ccda25] Correlation ID: 23ccda25, Job ID: 773cfe33-bbaa-49bc-bd2e-f443faed6657, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:30:01,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-55) [484ebcd6] Correlation ID: 484ebcd6, Job ID: 2ca9ca61-c6f2-4ac2-97d8-449e407047c6, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:30:01,824 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 2cc729e2 >2014-05-22 15:30:01,833 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 40310574 >2014-05-22 15:30:02,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) [23ccda25] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: 2cc729e2 >2014-05-22 15:30:02,180 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) [484ebcd6] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: 40310574 >2014-05-22 15:30:02,181 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-58) [23ccda25] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:30:02,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-55) [484ebcd6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:30:02,245 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-58) [44e3c346] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:30:02,247 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) [44e3c346] Host 52301919-41f3-4b25-9ebe-642d614ed5dd : 10.70.37.220 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:30:02,253 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-55) [45ac7c5f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:30:02,254 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) [45ac7c5f] Host 5f5c1a08-edd9-4de8-847e-701131ec8d30 : 10.70.37.217 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:30:14,323 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-11) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:30:14,789 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-85) No up server in cluster >2014-05-22 15:30:14,789 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-85) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:30:14,791 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-85) No up server in cluster >2014-05-22 15:30:14,791 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-85) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:31:14,796 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-27) No up server in cluster >2014-05-22 15:31:14,796 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-27) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:31:14,799 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-27) No up server in cluster >2014-05-22 15:31:14,799 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-27) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:32:14,803 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-69) No up server in cluster >2014-05-22 15:32:14,804 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-69) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:32:14,806 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-69) No up server in cluster >2014-05-22 15:32:14,806 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-69) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:33:01,440 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-41) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:33:14,810 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-11) No up server in cluster >2014-05-22 15:33:14,811 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-11) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:33:14,813 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-11) No up server in cluster >2014-05-22 15:33:14,813 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-11) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:33:31,835 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-7) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:34:02,226 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-39) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:34:11,382 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-47) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:34:14,396 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-48) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:34:14,818 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-58) [44e3c346] No up server in cluster >2014-05-22 15:34:14,818 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-58) [44e3c346] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:34:14,820 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-58) [44e3c346] No up server in cluster >2014-05-22 15:34:14,820 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-58) [44e3c346] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:35:00,023 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-67) Autorecovering 2 hosts >2014-05-22 15:35:00,023 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-67) Autorecovering hosts id: 52301919-41f3-4b25-9ebe-642d614ed5dd, name : 10.70.37.220 >2014-05-22 15:35:00,026 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:35:00,027 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:35:00,027 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Before acquiring lock in order to prevent monitoring for host 10.70.37.220 from data-center Default >2014-05-22 15:35:00,027 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.220 from data-center Default >2014-05-22 15:35:00,031 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7efa5da0 >2014-05-22 15:35:00,056 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] FINISH, SetVdsStatusVDSCommand, log id: 7efa5da0 >2014-05-22 15:35:00,065 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Activate finished. Lock released. Monitoring can run now for host 10.70.37.220 from data-center Default >2014-05-22 15:35:00,066 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:35:00,067 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-67) [2d0a2cc3] Autorecovering hosts id: 5f5c1a08-edd9-4de8-847e-701131ec8d30, name : 10.70.37.217 >2014-05-22 15:35:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:35:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:35:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Before acquiring lock in order to prevent monitoring for host 10.70.37.217 from data-center Default >2014-05-22 15:35:00,070 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.217 from data-center Default >2014-05-22 15:35:00,074 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4b7f4908 >2014-05-22 15:35:00,081 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] FINISH, SetVdsStatusVDSCommand, log id: 4b7f4908 >2014-05-22 15:35:00,089 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Activate finished. Lock released. Monitoring can run now for host 10.70.37.217 from data-center Default >2014-05-22 15:35:00,091 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-67) [5c2b3e76] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:35:03,005 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-36) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-05-22 15:35:03,063 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 2931b305 >2014-05-22 15:35:03,068 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: 1f097cd3 >2014-05-22 15:35:03,111 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GetHardwareInfoVDSCommand, log id: 2931b305 >2014-05-22 15:35:03,115 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GetHardwareInfoVDSCommand, log id: 1f097cd3 >2014-05-22 15:35:03,238 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:35:03,241 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 27fe06f >2014-05-22 15:35:03,247 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:35:03,249 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 226a53f7 >2014-05-22 15:35:03,255 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] FINISH, SetVdsStatusVDSCommand, log id: 27fe06f >2014-05-22 15:35:03,260 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:35:03,263 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] FINISH, SetVdsStatusVDSCommand, log id: 226a53f7 >2014-05-22 15:35:03,267 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:35:03,289 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) [73e3dde5] Correlation ID: 73e3dde5, Job ID: 40a26233-86cf-413d-8c51-3dc5f535760a, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:35:03,297 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) [de2dd7b] Correlation ID: de2dd7b, Job ID: bd68c945-594f-4879-829f-7762116362ec, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:35:03,323 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 46ffd61 >2014-05-22 15:35:03,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 14ed3a19 >2014-05-22 15:35:03,674 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) [73e3dde5] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: 46ffd61 >2014-05-22 15:35:03,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) [73e3dde5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:35:03,702 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) [de2dd7b] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: 14ed3a19 >2014-05-22 15:35:03,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) [de2dd7b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:35:03,752 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-71) [38f3861e] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:35:03,754 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) [38f3861e] Host 52301919-41f3-4b25-9ebe-642d614ed5dd : 10.70.37.220 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:35:03,775 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-72) [5f5aac2c] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:35:03,777 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) [5f5aac2c] Host 5f5c1a08-edd9-4de8-847e-701131ec8d30 : 10.70.37.217 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:35:14,825 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-100) No up server in cluster >2014-05-22 15:35:14,825 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-100) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:35:14,827 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-100) No up server in cluster >2014-05-22 15:35:14,828 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-100) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:36:14,831 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-38) No up server in cluster >2014-05-22 15:36:14,832 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-38) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:36:14,834 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-38) No up server in cluster >2014-05-22 15:36:14,834 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-38) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:37:14,838 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-86) No up server in cluster >2014-05-22 15:37:14,838 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-86) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:37:14,840 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-86) No up server in cluster >2014-05-22 15:37:14,840 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-86) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:38:14,845 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-25) No up server in cluster >2014-05-22 15:38:14,845 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-25) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:38:14,847 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-25) No up server in cluster >2014-05-22 15:38:14,848 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-25) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:39:14,851 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-68) No up server in cluster >2014-05-22 15:39:14,851 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-68) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:39:14,853 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-68) No up server in cluster >2014-05-22 15:39:14,854 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-68) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:40:00,020 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-79) Autorecovering 2 hosts >2014-05-22 15:40:00,020 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-79) Autorecovering hosts id: 52301919-41f3-4b25-9ebe-642d614ed5dd, name : 10.70.37.220 >2014-05-22 15:40:00,023 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Lock Acquired to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:40:00,023 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:40:00,023 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Before acquiring lock in order to prevent monitoring for host 10.70.37.220 from data-center Default >2014-05-22 15:40:00,023 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.220 from data-center Default >2014-05-22 15:40:00,029 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6f423e81 >2014-05-22 15:40:00,056 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] FINISH, SetVdsStatusVDSCommand, log id: 6f423e81 >2014-05-22 15:40:00,064 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Activate finished. Lock released. Monitoring can run now for host 10.70.37.220 from data-center Default >2014-05-22 15:40:00,066 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [515297e6] Lock freed to object EngineLock [exclusiveLocks= key: 52301919-41f3-4b25-9ebe-642d614ed5dd value: VDS >, sharedLocks= ] >2014-05-22 15:40:00,066 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-79) [515297e6] Autorecovering hosts id: 5f5c1a08-edd9-4de8-847e-701131ec8d30, name : 10.70.37.217 >2014-05-22 15:40:00,068 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Lock Acquired to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:40:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:40:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Before acquiring lock in order to prevent monitoring for host 10.70.37.217 from data-center Default >2014-05-22 15:40:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Lock acquired, from now a monitoring of host will be skipped for host 10.70.37.217 from data-center Default >2014-05-22 15:40:00,075 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 443c62aa >2014-05-22 15:40:00,089 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] FINISH, SetVdsStatusVDSCommand, log id: 443c62aa >2014-05-22 15:40:00,097 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Activate finished. Lock released. Monitoring can run now for host 10.70.37.217 from data-center Default >2014-05-22 15:40:00,099 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-79) [42ee69ba] Lock freed to object EngineLock [exclusiveLocks= key: 5f5c1a08-edd9-4de8-847e-701131ec8d30 value: VDS >, sharedLocks= ] >2014-05-22 15:40:01,400 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, vds=Host[10.70.37.220,52301919-41f3-4b25-9ebe-642d614ed5dd]), log id: 32498daa >2014-05-22 15:40:01,404 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GetHardwareInfoVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, vds=Host[10.70.37.217,5f5c1a08-edd9-4de8-847e-701131ec8d30]), log id: 499aefd1 >2014-05-22 15:40:01,448 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GetHardwareInfoVDSCommand, log id: 32498daa >2014-05-22 15:40:01,451 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GetHardwareInfoVDSCommand, log id: 499aefd1 >2014-05-22 15:40:01,530 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:40:01,533 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] START, SetVdsStatusVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 1e4bd4be >2014-05-22 15:40:01,539 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:40:01,541 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] START, SetVdsStatusVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 42b471c >2014-05-22 15:40:01,547 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] FINISH, SetVdsStatusVDSCommand, log id: 1e4bd4be >2014-05-22 15:40:01,550 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] Host 10.70.37.220 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:40:01,555 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] FINISH, SetVdsStatusVDSCommand, log id: 42b471c >2014-05-22 15:40:01,558 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] Host 10.70.37.217 is set to Non-Operational, it is missing the following networks: ovirtmgmt >2014-05-22 15:40:01,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) [68b64cd3] Correlation ID: 68b64cd3, Job ID: b9add559-8fdb-4775-b629-ce0f32c504e2, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.220 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:40:01,572 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) [210b4c6a] Correlation ID: 210b4c6a, Job ID: c905b691-a340-4356-bcc2-66b85abea459, Call Stack: null, Custom Event ID: -1, Message: Host 10.70.37.217 does not comply with the cluster test networks, the following networks are missing on host: 'ovirtmgmt' >2014-05-22 15:40:01,599 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] START, GlusterServersListVDSCommand(HostName = 10.70.37.220, HostId = 52301919-41f3-4b25-9ebe-642d614ed5dd), log id: 2c74f37a >2014-05-22 15:40:01,607 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] START, GlusterServersListVDSCommand(HostName = 10.70.37.217, HostId = 5f5c1a08-edd9-4de8-847e-701131ec8d30), log id: 1348488 >2014-05-22 15:40:01,924 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [68b64cd3] FINISH, GlusterServersListVDSCommand, return: [10.70.37.220:CONNECTED, 10.70.37.217:CONNECTED], log id: 2c74f37a >2014-05-22 15:40:01,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) [68b64cd3] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.220. >2014-05-22 15:40:01,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) [210b4c6a] FINISH, GlusterServersListVDSCommand, return: [10.70.37.217:CONNECTED, 10.70.37.220:CONNECTED], log id: 1348488 >2014-05-22 15:40:01,964 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) [210b4c6a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to NonOperational for host 10.70.37.217. >2014-05-22 15:40:02,002 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-80) [7adc97cb] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 52301919-41f3-4b25-9ebe-642d614ed5dd Type: VDS >2014-05-22 15:40:02,004 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) [7adc97cb] Host 52301919-41f3-4b25-9ebe-642d614ed5dd : 10.70.37.220 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:40:02,031 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-78) [3514f854] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5f5c1a08-edd9-4de8-847e-701131ec8d30 Type: VDS >2014-05-22 15:40:02,032 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) [3514f854] Host 5f5c1a08-edd9-4de8-847e-701131ec8d30 : 10.70.37.217 is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-05-22 15:40:14,857 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-11) No up server in cluster >2014-05-22 15:40:14,857 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-11) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:40:14,860 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-11) No up server in cluster >2014-05-22 15:40:14,860 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-11) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:40:38,400 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-7) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:41:14,864 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-51) [42da35c3] No up server in cluster >2014-05-22 15:41:14,865 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-51) [42da35c3] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:41:14,867 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-51) [42da35c3] No up server in cluster >2014-05-22 15:41:14,867 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-51) [42da35c3] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:41:39,183 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-7) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:42:14,871 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-91) No up server in cluster >2014-05-22 15:42:14,872 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-91) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:42:14,874 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-91) No up server in cluster >2014-05-22 15:42:14,874 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-91) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:42:52,108 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-22) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-05-22 15:43:14,878 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-41) No up server in cluster >2014-05-22 15:43:14,879 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-41) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:43:14,881 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-41) No up server in cluster >2014-05-22 15:43:14,881 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-41) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:84) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) [:1.7.0_55] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-05-22 15:43:25,497 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-4-thread-45) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue.
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1100219
: 898316 |
898317