Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 836013 Details for
Bug 1041857
rest api: rebalance causes hosts to become Non Operational after adding 4 bricks sequentially to a 6 node dist volume
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
engine.log
engine.log (text/x-log), 1.16 MB, created by
Dustin Tsang
on 2013-12-12 19:51:00 UTC
(
hide
)
Description:
engine.log
Filename:
MIME Type:
Creator:
Dustin Tsang
Created:
2013-12-12 19:51:00 UTC
Size:
1.16 MB
patch
obsolete
>2013-12-12 12:09:51,457 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2013-12-12 12:09:51,463 WARN [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2013-12-12 12:09:51,464 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhsc-doc.conf". >2013-12-12 12:09:51,464 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2013-12-12 12:09:51,464 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2013-12-12 12:09:51,464 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2013-12-12 12:09:51,464 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2013-12-12 12:09:51,465 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-access-plugin.conf". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_AJP_ENABLED" is "true". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_AJP_PORT" is "8702". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_APPS" is "engine.ear redhat-access-plugin-rhsc.ear". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_DATABASE" is "engine". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2013-12-12 12:09:51,472 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_HOST" is "localhost". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_PASSWORD" is "***". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_PORT" is "5432". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_SECURED" is "False". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_SSL" is "false". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DB_USER" is "engine". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2013-12-12 12:09:51,473 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_FQDN" is "latest.rhsc". >2013-12-12 12:09:51,474 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_GROUP" is "ovirt". >2013-12-12 12:09:51,474 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HEAP_MAX" is "1g". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HEAP_MIN" is "1g". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HTTPS_PORT" is "None". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HTTP_ENABLED" is "false". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_HTTP_PORT" is "None". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhsc-doc/manual". >2013-12-12 12:09:51,482 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PERM_MAX" is "256m". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PERM_MIN" is "256m". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PROXY_ENABLED" is "true". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_STOP_INTERVAL" is "1". >2013-12-12 12:09:51,483 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_STOP_TIME" is "10". >2013-12-12 12:09:51,484 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2013-12-12 12:09:51,484 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_USER" is "ovirt". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "ENGINE_VERBOSE_GC" is "false". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2013-12-12 12:09:51,492 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 36) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2013-12-12 12:09:53,170 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 38) Start initializing Backend >2013-12-12 12:09:53,910 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 38) Running ovirt-engine 2.1.2-0.27.beta.el6_5 >2013-12-12 12:09:53,912 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 38) Start initializing dictionaries >2013-12-12 12:09:53,924 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 38) Finished initializing dictionaries >2013-12-12 12:09:53,925 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 38) Start initializing AuditLogCleanupManager >2013-12-12 12:09:53,925 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 38) Setting audit cleanup manager to run at: 35 35 3 * * ? >2013-12-12 12:09:53,966 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 38) Finished initializing AuditLogCleanupManager >2013-12-12 12:09:53,972 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 38) Start initializing TagsDirector >2013-12-12 12:09:53,995 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 38) Tag root added to tree >2013-12-12 12:09:54,023 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 38) Finished initializing TagsDirector >2013-12-12 12:09:54,025 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 38) Start initializing IsoDomainListSyncronizer >2013-12-12 12:09:54,040 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 38) Finished initializing IsoDomainListSyncronizer >2013-12-12 12:09:54,058 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 38) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2013-12-12 12:09:54,706 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 38) Completed initializing handlers >2013-12-12 12:09:54,708 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 38) Start initializing ErrorTranslatorImpl >2013-12-12 12:09:54,737 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 38) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-12-12 12:09:54,743 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 38) Finished initializing ErrorTranslatorImpl >2013-12-12 12:09:54,743 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 38) Start initializing ErrorTranslatorImpl >2013-12-12 12:09:54,744 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 38) Finished initializing ErrorTranslatorImpl >2013-12-12 12:09:54,744 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 38) Mark incomplete jobs as UNKNOWN >2013-12-12 12:09:54,804 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 38) Start initializing JobRepositoryCleanupManager >2013-12-12 12:09:54,809 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 38) Finished initializing JobRepositoryCleanupManager >2013-12-12 12:09:54,810 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 38) Start initializing AutoRecoveryManager >2013-12-12 12:09:54,817 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 38) Finished initializing AutoRecoveryManager >2013-12-12 12:09:54,819 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 38) Start initializing ExecutionMessageDirector >2013-12-12 12:09:54,822 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 38) Finished initializing ExecutionMessageDirector >2013-12-12 12:09:54,866 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 45) Start initializing UsersDomainsCacheManagerService >2013-12-12 12:09:54,890 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 45) Start initializing DbUserCacheManager >2013-12-12 12:09:54,893 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 45) Finished initializing DbUserCacheManager >2013-12-12 12:09:54,893 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 45) Finished initializing UsersDomainsCacheManagerService >2013-12-12 12:09:54,916 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ServerService Thread Pool -- 37) Initialization of AsyncTaskManager completed successfully. >2013-12-12 12:09:54,923 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 37) Start initializing ResourceManager >2013-12-12 12:09:54,938 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-12-12 12:09:55,179 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 37) Finished initializing ResourceManager >2013-12-12 12:09:55,190 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 37) Initialization of OvfDataUpdater completed successfully. >2013-12-12 12:09:55,193 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 37) Start scheduling to enable vds load balancer >2013-12-12 12:09:55,193 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 37) Finished scheduling to enable vds load balancer >2013-12-12 12:09:55,219 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-4-thread-1) MacPoolManager(3f2b5cce): Start initializing >2013-12-12 12:09:55,241 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 37) Init VM custom properties utilities >2013-12-12 12:09:55,289 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-4-thread-1) MacPoolManager(3f2b5cce): Finished initializing >2013-12-12 12:09:55,328 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 37) Init device custom properties utilities >2013-12-12 12:09:55,339 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 37) Initializing Scheduling manager >2013-12-12 12:09:55,425 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 37) External scheduler disabled, discovery skipped >2013-12-12 12:09:55,425 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 37) Initialized Scheduling manager >2013-12-12 12:09:55,426 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 37) Initializing DWH Heart Beat >2013-12-12 12:10:55,317 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-44) No up server in cluster >2013-12-12 12:10:55,334 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-44) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:11:55,343 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-88) No up server in cluster >2013-12-12 12:11:55,343 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-88) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:12:55,346 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-30) No up server in cluster >2013-12-12 12:12:55,346 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-30) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:13:55,348 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-73) No up server in cluster >2013-12-12 12:13:55,348 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-73) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:14:55,352 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-19) No up server in cluster >2013-12-12 12:14:55,352 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-19) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:15:55,355 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-63) No up server in cluster >2013-12-12 12:15:55,355 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-63) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:16:55,358 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2013-12-12 12:16:55,358 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:17:55,361 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-49) No up server in cluster >2013-12-12 12:17:55,361 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-49) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:18:55,367 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-92) No up server in cluster >2013-12-12 12:18:55,367 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-92) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:19:55,370 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-38) No up server in cluster >2013-12-12 12:19:55,370 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-38) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:20:55,372 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-84) No up server in cluster >2013-12-12 12:20:55,372 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-84) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:21:55,375 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-27) No up server in cluster >2013-12-12 12:21:55,375 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-27) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:22:55,379 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-69) No up server in cluster >2013-12-12 12:22:55,379 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-69) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:23:55,381 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-13) No up server in cluster >2013-12-12 12:23:55,381 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-13) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:24:55,386 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-57) No up server in cluster >2013-12-12 12:24:55,386 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-57) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:25:55,389 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-1) No up server in cluster >2013-12-12 12:25:55,389 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-1) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:26:55,391 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-45) No up server in cluster >2013-12-12 12:26:55,391 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-45) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:27:55,394 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-86) No up server in cluster >2013-12-12 12:27:55,394 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-86) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:28:55,396 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-29) No up server in cluster >2013-12-12 12:28:55,397 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-29) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:29:55,399 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-75) No up server in cluster >2013-12-12 12:29:55,399 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-75) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:30:55,404 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-20) No up server in cluster >2013-12-12 12:30:55,404 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-20) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:31:55,407 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-63) No up server in cluster >2013-12-12 12:31:55,407 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-63) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:32:55,409 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2013-12-12 12:32:55,409 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:33:55,411 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-49) No up server in cluster >2013-12-12 12:33:55,412 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-49) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:34:55,414 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-94) No up server in cluster >2013-12-12 12:34:55,414 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-94) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:35:55,419 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-38) No up server in cluster >2013-12-12 12:35:55,419 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-38) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:36:55,423 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-82) No up server in cluster >2013-12-12 12:36:55,424 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-82) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:37:55,427 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-24) No up server in cluster >2013-12-12 12:37:55,427 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-24) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:38:55,429 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-67) No up server in cluster >2013-12-12 12:38:55,429 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-67) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:39:55,434 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 12:39:55,434 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:40:55,436 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-59) No up server in cluster >2013-12-12 12:40:55,436 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-59) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:41:55,439 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2013-12-12 12:41:55,439 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:42:55,442 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-44) No up server in cluster >2013-12-12 12:42:55,442 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-44) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:43:55,445 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-88) No up server in cluster >2013-12-12 12:43:55,445 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-88) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:44:55,447 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-33) No up server in cluster >2013-12-12 12:44:55,448 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-33) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:45:55,450 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-77) No up server in cluster >2013-12-12 12:45:55,450 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-77) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:46:55,456 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-20) No up server in cluster >2013-12-12 12:46:55,456 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-20) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:47:55,459 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-63) No up server in cluster >2013-12-12 12:47:55,459 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-63) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:48:12,019 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-6) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc.json] >2013-12-12 12:48:12,029 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-6) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc-config.json] >2013-12-12 12:48:19,050 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [42579fd5] Running command: LogoutUserCommand internal: false. >2013-12-12 12:48:19,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: UNASSIGNED not exist in string table >2013-12-12 12:48:19,093 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: DWH_STOPPED not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: DWH_STARTED not exist in string table >2013-12-12 12:48:19,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: DWH_ERROR not exist in string table >2013-12-12 12:48:19,117 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2013-12-12 12:48:19,120 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: GLUSTER_VOLUME_REBALANCE_START_DETECTED_FROM_CLI not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: START_REMOVING_GLUSTER_VOLUME_BRICKS_DETECTED_FROM_CLI not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2013-12-12 12:48:19,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2013-12-12 12:48:19,122 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2013-12-12 12:48:19,122 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2013-12-12 12:48:19,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [42579fd5] Correlation ID: 42579fd5, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 12:48:31,680 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginAdminUserCommand internal: false. >2013-12-12 12:48:31,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 12:48:34,248 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 12:48:34,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 12:48:55,462 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-6) No up server in cluster >2013-12-12 12:48:55,462 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-6) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:49:55,466 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-52) No up server in cluster >2013-12-12 12:49:55,466 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-52) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:50:37,450 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-5) [3a903201] Running command: AddVdsCommand internal: false. Entities affected : ID: 99408929-82cf-4dc7-a532-9d998063fa95 Type: VdsGroups >2013-12-12 12:50:37,610 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 12:50:37,633 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 12:50:37,640 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] Cannot get vdsManager for vdsid=81d38815-c70e-4edd-bfbe-9b854d37252d >2013-12-12 12:50:37,665 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] START, RemoveVdsVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 25bee421 >2013-12-12 12:50:37,665 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] Cannot get vdsManager for vdsid=81d38815-c70e-4edd-bfbe-9b854d37252d >2013-12-12 12:50:37,665 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] FINISH, RemoveVdsVDSCommand, log id: 25bee421 >2013-12-12 12:50:37,666 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] Cannot get vdsManager for vdsid=81d38815-c70e-4edd-bfbe-9b854d37252d >2013-12-12 12:50:37,670 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] START, AddVdsVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7637a765 >2013-12-12 12:50:37,670 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] AddVds - entered , starting logic to add VDS 81d38815-c70e-4edd-bfbe-9b854d37252d >2013-12-12 12:50:37,673 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] AddVds - VDS 81d38815-c70e-4edd-bfbe-9b854d37252d was added, will try to add it to the resource manager >2013-12-12 12:50:37,676 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] Entered VdsManager constructor >2013-12-12 12:50:37,712 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] Initialize vdsBroker (latest-a,54,321) >2013-12-12 12:50:37,997 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-5) [2f1e0174] VDS 81d38815-c70e-4edd-bfbe-9b854d37252d was added to the Resource Manager >2013-12-12 12:50:37,998 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [2f1e0174] FINISH, AddVdsVDSCommand, log id: 7637a765 >2013-12-12 12:50:38,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [2f1e0174] Correlation ID: 3a903201, Job ID: 013d9a23-5d9c-4dcd-80ce-5151e2e6499d, Call Stack: null, Custom Event ID: -1, Message: Host latest-a was added by admin@internal. >2013-12-12 12:50:38,061 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-4) [7e91f840] Lock Acquired to object EngineLock [exclusiveLocks= key: 81d38815-c70e-4edd-bfbe-9b854d37252d value: VDS >, sharedLocks= ] >2013-12-12 12:50:38,083 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-4) [7e91f840] Running command: InstallVdsCommand internal: true. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 12:50:38,083 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-4) [7e91f840] Before Installation host 81d38815-c70e-4edd-bfbe-9b854d37252d, latest-a >2013-12-12 12:50:38,125 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-4) [7e91f840] START, SetVdsStatusVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, status=Installing, nonOperationalReason=NONE), log id: 7b16c13f >2013-12-12 12:50:38,150 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-4) [7e91f840] FINISH, SetVdsStatusVDSCommand, log id: 7b16c13f >2013-12-12 12:50:38,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-4) [7e91f840] Installation latest-a: Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 12:50:38,266 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-4) [7e91f840] Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 12:50:38,284 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-4) [7e91f840] Installation of latest-a. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 12:50:38,284 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (pool-4-thread-4) Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' is missing, creating >2013-12-12 12:50:38,481 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-4) SSH execute root@latest-a 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 12:50:39,291 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Initializing >2013-12-12 12:50:39,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Initializing. >2013-12-12 12:50:39,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment setup >2013-12-12 12:50:39,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Environment setup. >2013-12-12 12:50:39,365 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment packages setup >2013-12-12 12:50:39,375 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Environment packages setup. >2013-12-12 12:50:41,002 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Initializing Host: latest-a >2013-12-12 12:50:42,182 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Programs detection >2013-12-12 12:50:42,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Programs detection. >2013-12-12 12:50:42,191 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment customization >2013-12-12 12:50:42,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Environment customization. >2013-12-12 12:50:42,226 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Logs at host located at: '/tmp/ovirt-host-deploy-20131212125039.log' >2013-12-12 12:50:42,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Logs at host located at: '/tmp/ovirt-host-deploy-20131212125039.log'. >2013-12-12 12:50:42,375 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-a reports unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 >2013-12-12 12:50:42,403 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 to Host latest-a >2013-12-12 12:50:42,715 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Setup validation >2013-12-12 12:50:42,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Setup validation. >2013-12-12 12:50:42,889 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a' >2013-12-12 12:50:42,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Host latest-a installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a'. >2013-12-12 12:50:42,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction setup >2013-12-12 12:50:42,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Transaction setup. >2013-12-12 12:50:42,926 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 12:50:42,941 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Misc configuration. >2013-12-12 12:50:42,942 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Package installation >2013-12-12 12:50:42,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Package installation. >2013-12-12 12:50:44,519 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 12:50:44,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Misc configuration. >2013-12-12 12:50:44,576 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Setting time >2013-12-12 12:50:44,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Setting time. >2013-12-12 12:50:47,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Enrolling certificate >2013-12-12 12:50:47,861 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Enrolling certificate. >2013-12-12 12:50:49,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction commit >2013-12-12 12:50:49,283 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Transaction commit. >2013-12-12 12:50:49,383 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Closing up >2013-12-12 12:50:49,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Closing up. >2013-12-12 12:50:50,022 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting gluster >2013-12-12 12:50:50,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Starting gluster. >2013-12-12 12:50:55,469 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2013-12-12 12:50:55,469 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:50:57,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting vdsm >2013-12-12 12:50:58,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Starting vdsm. >2013-12-12 12:51:01,322 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Pre-termination >2013-12-12 12:51:01,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Pre-termination. >2013-12-12 12:51:02,367 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212125102-latest-a-7e91f840.log' >2013-12-12 12:51:02,374 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212125102-latest-a-7e91f840.log'. >2013-12-12 12:51:03,350 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Termination >2013-12-12 12:51:03,726 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 7e91f840, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-a. Stage: Termination. >2013-12-12 12:51:04,089 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-4) [7e91f840] Engine managed to communicate with VDSM agent on host latest-a >2013-12-12 12:51:04,519 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-4) [7e91f840] START, SetVdsStatusVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, status=Initializing, nonOperationalReason=NONE), log id: 24db98ae >2013-12-12 12:51:04,534 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-4) [7e91f840] FINISH, SetVdsStatusVDSCommand, log id: 24db98ae >2013-12-12 12:51:04,534 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-4) [7e91f840] After Installation host latest-a, VDS >2013-12-12 12:51:04,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-4) [7e91f840] Correlation ID: 7e91f840, Job ID: 013d9a23-5d9c-4dcd-80ce-5151e2e6499d, Call Stack: null, Custom Event ID: -1, Message: Host latest-a installed >2013-12-12 12:51:04,576 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-4) [7e91f840] Lock freed to object EngineLock [exclusiveLocks= key: 81d38815-c70e-4edd-bfbe-9b854d37252d value: VDS >, sharedLocks= ] >2013-12-12 12:51:05,120 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GetHardwareInfoVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, vds=Host[latest-a]), log id: 2092a7f5 >2013-12-12 12:51:05,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GetHardwareInfoVDSCommand, log id: 2092a7f5 >2013-12-12 12:51:05,236 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5e411f24 >2013-12-12 12:51:05,647 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 5e411f24 >2013-12-12 12:51:05,773 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GetHardwareInfoVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, vds=Host[latest-a]), log id: 20897ed8 >2013-12-12 12:51:05,819 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GetHardwareInfoVDSCommand, log id: 20897ed8 >2013-12-12 12:51:05,845 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3653dffe >2013-12-12 12:51:06,339 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 3653dffe >2013-12-12 12:51:06,351 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host latest-a. >2013-12-12 12:51:06,394 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-14) [55b6f83d] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 12:51:06,403 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-14) [55b6f83d] START, GetGlusterHostUUIDVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6dc34ebb >2013-12-12 12:51:06,606 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-14) [55b6f83d] FINISH, GetGlusterHostUUIDVDSCommand, return: 76102dca-bdfd-448b-a1cc-7a967cf84589, log id: 6dc34ebb >2013-12-12 12:51:06,744 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 12:51:07,117 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7385b5b4 >2013-12-12 12:51:07,507 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 7385b5b4 >2013-12-12 12:51:07,511 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1863da62 >2013-12-12 12:51:07,726 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-16) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:07,727 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-16) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:07,728 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-16) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:07,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@abbe6a09}, log id: 1863da62 >2013-12-12 12:51:07,739 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-16) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:12,762 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 14524107 >2013-12-12 12:51:13,361 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 14524107 >2013-12-12 12:51:13,363 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 305f573e >2013-12-12 12:51:13,560 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-19) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:13,563 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-19) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:13,564 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-19) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:13,564 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c2dee74e}, log id: 305f573e >2013-12-12 12:51:13,569 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-19) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:18,577 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 43d39 >2013-12-12 12:51:19,204 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 43d39 >2013-12-12 12:51:19,206 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2371da3f >2013-12-12 12:51:19,411 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-25) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:19,412 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-25) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:19,413 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-25) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:19,413 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fcb294a1}, log id: 2371da3f >2013-12-12 12:51:19,417 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-25) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:24,447 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 46cffc74 >2013-12-12 12:51:24,969 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:DISCONNECTED], log id: 46cffc74 >2013-12-12 12:51:24,971 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1fea4f9 >2013-12-12 12:51:25,179 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-29) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:25,180 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-29) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:25,181 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-29) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:25,181 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4cf1120b}, log id: 1fea4f9 >2013-12-12 12:51:25,185 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-29) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:30,194 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: e5d7fa4 >2013-12-12 12:51:30,972 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: e5d7fa4 >2013-12-12 12:51:30,973 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3755a07c >2013-12-12 12:51:31,192 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-37) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:31,193 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-37) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:31,194 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-37) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:31,198 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5c31d2cf}, log id: 3755a07c >2013-12-12 12:51:31,199 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-37) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:36,204 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3e1e1baf >2013-12-12 12:51:36,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3e1e1baf >2013-12-12 12:51:36,706 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: ef13424 >2013-12-12 12:51:36,891 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-45) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:36,892 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-45) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:36,893 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-45) Could not find server latest-b in cluster 99408929-82cf-4dc7-a532-9d998063fa95 >2013-12-12 12:51:36,893 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ca0ac3fb}, log id: ef13424 >2013-12-12 12:51:36,894 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-45) Volume blah contains brick(s) from unknown hosts. Hence will not add it to engine at this point. >2013-12-12 12:51:38,380 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-11) [7f0423ce] Running command: AddVdsCommand internal: false. Entities affected : ID: 99408929-82cf-4dc7-a532-9d998063fa95 Type: VdsGroups >2013-12-12 12:51:38,407 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 12:51:38,413 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 12:51:38,413 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Cannot get vdsManager for vdsid=d43e3091-09a3-4e1b-b07e-8a7272692b99 >2013-12-12 12:51:38,415 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] START, RemoveVdsVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 64c7d967 >2013-12-12 12:51:38,415 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Cannot get vdsManager for vdsid=d43e3091-09a3-4e1b-b07e-8a7272692b99 >2013-12-12 12:51:38,415 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] FINISH, RemoveVdsVDSCommand, log id: 64c7d967 >2013-12-12 12:51:38,415 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Cannot get vdsManager for vdsid=d43e3091-09a3-4e1b-b07e-8a7272692b99 >2013-12-12 12:51:38,416 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] START, AddVdsVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 384d57ca >2013-12-12 12:51:38,416 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] AddVds - entered , starting logic to add VDS d43e3091-09a3-4e1b-b07e-8a7272692b99 >2013-12-12 12:51:38,417 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] AddVds - VDS d43e3091-09a3-4e1b-b07e-8a7272692b99 was added, will try to add it to the resource manager >2013-12-12 12:51:38,417 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Entered VdsManager constructor >2013-12-12 12:51:38,451 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Initialize vdsBroker (latest-b,54,321) >2013-12-12 12:51:38,452 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-11) [3c18f3f7] VDS d43e3091-09a3-4e1b-b07e-8a7272692b99 was added to the Resource Manager >2013-12-12 12:51:38,452 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-11) [3c18f3f7] FINISH, AddVdsVDSCommand, log id: 384d57ca >2013-12-12 12:51:38,488 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [3c18f3f7] Correlation ID: 7f0423ce, Job ID: 946d3a48-8437-4a5b-9eb9-7c4166d9a3c0, Call Stack: null, Custom Event ID: -1, Message: Host latest-b was added by admin@internal. >2013-12-12 12:51:38,500 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-40) [2710cde3] Lock Acquired to object EngineLock [exclusiveLocks= key: d43e3091-09a3-4e1b-b07e-8a7272692b99 value: VDS >, sharedLocks= ] >2013-12-12 12:51:38,521 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-40) [2710cde3] Running command: InstallVdsCommand internal: true. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 12:51:38,521 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-40) [2710cde3] Before Installation host d43e3091-09a3-4e1b-b07e-8a7272692b99, latest-b >2013-12-12 12:51:38,524 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-40) [2710cde3] START, SetVdsStatusVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, status=Installing, nonOperationalReason=NONE), log id: 553fff >2013-12-12 12:51:38,538 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-40) [2710cde3] FINISH, SetVdsStatusVDSCommand, log id: 553fff >2013-12-12 12:51:38,616 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-40) [2710cde3] Installation latest-b: Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 12:51:38,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [2710cde3] Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 12:51:38,648 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-40) [2710cde3] Installation of latest-b. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 12:51:38,655 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-40) SSH execute root@latest-b 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 12:51:39,317 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Initializing >2013-12-12 12:51:39,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Initializing. >2013-12-12 12:51:39,465 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment setup >2013-12-12 12:51:39,479 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Environment setup. >2013-12-12 12:51:39,482 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment packages setup >2013-12-12 12:51:39,496 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Environment packages setup. >2013-12-12 12:51:40,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Programs detection >2013-12-12 12:51:40,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Programs detection. >2013-12-12 12:51:40,672 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment customization >2013-12-12 12:51:40,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Environment customization. >2013-12-12 12:51:40,696 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Logs at host located at: '/tmp/ovirt-host-deploy-20131212125139.log' >2013-12-12 12:51:40,705 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Logs at host located at: '/tmp/ovirt-host-deploy-20131212125139.log'. >2013-12-12 12:51:40,769 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-b reports unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 >2013-12-12 12:51:40,800 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 to Host latest-b >2013-12-12 12:51:40,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Setup validation >2013-12-12 12:51:40,913 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Setup validation. >2013-12-12 12:51:41,080 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b' >2013-12-12 12:51:41,088 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Host latest-b installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b'. >2013-12-12 12:51:41,103 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction setup >2013-12-12 12:51:41,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Transaction setup. >2013-12-12 12:51:41,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 12:51:41,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Misc configuration. >2013-12-12 12:51:41,327 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Package installation >2013-12-12 12:51:41,355 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Package installation. >2013-12-12 12:51:41,454 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Initializing Host: latest-b >2013-12-12 12:51:41,905 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 28c7638c >2013-12-12 12:51:42,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 28c7638c >2013-12-12 12:51:42,357 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 690d321b >2013-12-12 12:51:42,560 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@88bc715c}, log id: 690d321b >2013-12-12 12:51:42,579 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected new volume blah on cluster Default, and added it to engine DB. >2013-12-12 12:51:44,942 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 12:51:44,955 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Misc configuration. >2013-12-12 12:51:44,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Setting time >2013-12-12 12:51:44,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Setting time. >2013-12-12 12:51:47,718 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 220808f8 >2013-12-12 12:51:48,136 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 220808f8 >2013-12-12 12:51:48,138 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3eabf427 >2013-12-12 12:51:48,491 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@586088bb}, log id: 3eabf427 >2013-12-12 12:51:48,651 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Enrolling certificate >2013-12-12 12:51:48,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Enrolling certificate. >2013-12-12 12:51:50,099 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction commit >2013-12-12 12:51:50,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Transaction commit. >2013-12-12 12:51:50,196 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Closing up >2013-12-12 12:51:50,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Closing up. >2013-12-12 12:51:50,381 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting gluster >2013-12-12 12:51:50,405 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Starting gluster. >2013-12-12 12:51:53,528 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 120a1117 >2013-12-12 12:51:53,979 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 120a1117 >2013-12-12 12:51:53,980 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2e456ebb >2013-12-12 12:51:54,290 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c9e0109b}, log id: 2e456ebb >2013-12-12 12:51:55,474 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 13b101c5 >2013-12-12 12:51:58,305 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterTasksListVDSCommand, log id: 13b101c5 >2013-12-12 12:51:58,305 ERROR [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-68) org.ovirt.engine.core.common.errors.VDSError@645ddd7b >2013-12-12 12:51:58,305 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-68) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: Command execution failed >return code: 2 (Failed with error GlusterVolumeStatusAllFailedException and code 4161) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:51:59,303 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4c8ad91a >2013-12-12 12:51:59,544 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting vdsm >2013-12-12 12:51:59,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Starting vdsm. >2013-12-12 12:51:59,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4c8ad91a >2013-12-12 12:51:59,698 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6a16ae1e >2013-12-12 12:51:59,908 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1c5306d3}, log id: 6a16ae1e >2013-12-12 12:52:03,076 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Pre-termination >2013-12-12 12:52:03,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Pre-termination. >2013-12-12 12:52:04,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212125204-latest-b-2710cde3.log' >2013-12-12 12:52:04,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212125204-latest-b-2710cde3.log'. >2013-12-12 12:52:04,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Termination >2013-12-12 12:52:04,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 2710cde3, Call Stack: null, Custom Event ID: -1, Message: Installing Host latest-b. Stage: Termination. >2013-12-12 12:52:04,925 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5b566087 >2013-12-12 12:52:05,123 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-40) [2710cde3] Engine managed to communicate with VDSM agent on host latest-b >2013-12-12 12:52:05,265 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5b566087 >2013-12-12 12:52:05,266 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: cdaef6d >2013-12-12 12:52:05,297 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-40) [2710cde3] START, SetVdsStatusVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, status=Initializing, nonOperationalReason=NONE), log id: 2257cc63 >2013-12-12 12:52:05,304 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-40) [2710cde3] FINISH, SetVdsStatusVDSCommand, log id: 2257cc63 >2013-12-12 12:52:05,304 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-40) [2710cde3] After Installation host latest-b, VDS >2013-12-12 12:52:05,338 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [2710cde3] Correlation ID: 2710cde3, Job ID: 946d3a48-8437-4a5b-9eb9-7c4166d9a3c0, Call Stack: null, Custom Event ID: -1, Message: Host latest-b installed >2013-12-12 12:52:05,363 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-40) [2710cde3] Lock freed to object EngineLock [exclusiveLocks= key: d43e3091-09a3-4e1b-b07e-8a7272692b99 value: VDS >, sharedLocks= ] >2013-12-12 12:52:05,428 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8c2485c5}, log id: cdaef6d >2013-12-12 12:52:05,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GetHardwareInfoVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, vds=Host[latest-b]), log id: 4b07e991 >2013-12-12 12:52:05,566 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GetHardwareInfoVDSCommand, log id: 4b07e991 >2013-12-12 12:52:05,590 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2f9d0c50 >2013-12-12 12:52:06,333 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2f9d0c50 >2013-12-12 12:52:06,373 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GetHardwareInfoVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, vds=Host[latest-b]), log id: 3e09c5c8 >2013-12-12 12:52:06,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GetHardwareInfoVDSCommand, log id: 3e09c5c8 >2013-12-12 12:52:06,406 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1edae9b0 >2013-12-12 12:52:06,796 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1edae9b0 >2013-12-12 12:52:06,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host latest-b. >2013-12-12 12:52:06,812 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-84) [2b5c035b] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 12:52:06,813 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-84) [2b5c035b] START, GetGlusterHostUUIDVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7e7ad6ae >2013-12-12 12:52:07,073 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-84) [2b5c035b] FINISH, GetGlusterHostUUIDVDSCommand, return: 17277992-0ec5-4f41-9f13-3360571eee4e, log id: 7e7ad6ae >2013-12-12 12:52:07,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [2b5c035b] START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6aebb1af >2013-12-12 12:52:07,587 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [2b5c035b] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6aebb1af >2013-12-12 12:52:07,608 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 12:52:10,445 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 532f571b >2013-12-12 12:52:10,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 532f571b >2013-12-12 12:52:10,901 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3e8bbe7d >2013-12-12 12:52:11,118 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bbcc56e4}, log id: 3e8bbe7d >2013-12-12 12:52:16,133 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 60fbc5b4 >2013-12-12 12:52:16,559 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 60fbc5b4 >2013-12-12 12:52:16,561 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7a0f5adc >2013-12-12 12:52:16,874 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3bcc3c35}, log id: 7a0f5adc >2013-12-12 12:52:21,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6dbc6d71 >2013-12-12 12:52:22,285 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6dbc6d71 >2013-12-12 12:52:22,287 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3ba79432 >2013-12-12 12:52:22,523 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4e93dabb}, log id: 3ba79432 >2013-12-12 12:52:27,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: de963b7 >2013-12-12 12:52:27,587 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-5) [5306ad2a] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:52:27,916 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: de963b7 >2013-12-12 12:52:27,917 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-5) [5306ad2a] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:52:27,918 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 74f3efa0 >2013-12-12 12:52:28,096 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterVolumesListVDSCommand, return: {1bb7cfaf-2110-4058-96cd-58cc7caacec0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1f9d03fa}, log id: 74f3efa0 >2013-12-12 12:52:28,269 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-40) [5306ad2a] Running command: DeleteGlusterVolumeCommand internal: false. Entities affected : ID: 1bb7cfaf-2110-4058-96cd-58cc7caacec0 Type: GlusterVolume >2013-12-12 12:52:28,272 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-40) [5306ad2a] START, DeleteGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4c19d23b >2013-12-12 12:52:28,571 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-40) [5306ad2a] FINISH, DeleteGlusterVolumeVDSCommand, log id: 4c19d23b >2013-12-12 12:52:28,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [5306ad2a] Correlation ID: 5306ad2a, Job ID: f680516b-4ea7-4980-8ca5-d780383fb712, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume blah deleted. >2013-12-12 12:52:28,618 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-40) [5306ad2a] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:52:33,110 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-18) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 449d3e97 >2013-12-12 12:52:33,700 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 449d3e97 >2013-12-12 12:52:33,702 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-18) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5ea651cd >2013-12-12 12:52:33,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 5ea651cd >2013-12-12 12:52:38,895 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2779fea >2013-12-12 12:52:39,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2779fea >2013-12-12 12:52:39,349 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 387f7bb5 >2013-12-12 12:52:39,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 387f7bb5 >2013-12-12 12:52:44,665 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 512cbb85 >2013-12-12 12:52:45,151 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 512cbb85 >2013-12-12 12:52:45,153 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5d761561 >2013-12-12 12:52:45,333 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 5d761561 >2013-12-12 12:52:50,339 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4f5233a1 >2013-12-12 12:52:50,694 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4f5233a1 >2013-12-12 12:52:50,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 58d01b69 >2013-12-12 12:52:50,975 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 58d01b69 >2013-12-12 12:52:55,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: ec6ee86 >2013-12-12 12:52:56,347 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: ec6ee86 >2013-12-12 12:52:56,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5fdad390 >2013-12-12 12:52:56,590 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 5fdad390 >2013-12-12 12:52:58,309 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-54) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 905fcc9 >2013-12-12 12:52:58,497 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, GlusterTasksListVDSCommand, return: [], log id: 905fcc9 >2013-12-12 12:53:01,598 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5ed1bbd7 >2013-12-12 12:53:01,964 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5ed1bbd7 >2013-12-12 12:53:01,966 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6b615835 >2013-12-12 12:53:02,283 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6b615835 >2013-12-12 12:53:07,292 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 8f5c765 >2013-12-12 12:53:07,642 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 8f5c765 >2013-12-12 12:53:07,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 411e87f5 >2013-12-12 12:53:07,845 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 411e87f5 >2013-12-12 12:53:12,852 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5758ca26 >2013-12-12 12:53:13,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5758ca26 >2013-12-12 12:53:13,214 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 24d9de10 >2013-12-12 12:53:13,535 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 24d9de10 >2013-12-12 12:53:18,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 461b24c8 >2013-12-12 12:53:18,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 461b24c8 >2013-12-12 12:53:18,934 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6ee6824e >2013-12-12 12:53:19,121 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6ee6824e >2013-12-12 12:53:24,130 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 649aa432 >2013-12-12 12:53:24,529 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 649aa432 >2013-12-12 12:53:24,531 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6230230e >2013-12-12 12:53:24,719 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6230230e >2013-12-12 12:53:29,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2c9b6c34 >2013-12-12 12:53:30,398 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2c9b6c34 >2013-12-12 12:53:30,400 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 161a7e30 >2013-12-12 12:53:30,575 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 161a7e30 >2013-12-12 12:53:31,819 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [4d06fb31] Running command: CreateGlusterVolumeCommand internal: false. Entities affected : ID: 99408929-82cf-4dc7-a532-9d998063fa95 Type: VdsGroups >2013-12-12 12:53:31,821 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) [4d06fb31] START, CreateGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3442348f >2013-12-12 12:53:31,996 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) [4d06fb31] FINISH, CreateGlusterVolumeVDSCommand, log id: 3442348f >2013-12-12 12:53:32,019 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [4d06fb31] Correlation ID: 4d06fb31, Job ID: 89383401-0664-4328-abd0-b8a2bf5dd9c0, Call Stack: null, Custom Event ID: -1, Message: Creation of Gluster Volume boom failed. >2013-12-12 12:53:32,028 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [4d06fb31] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:53:35,581 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4e61f323 >2013-12-12 12:53:36,131 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4e61f323 >2013-12-12 12:53:36,133 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 464b3586 >2013-12-12 12:53:36,474 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 464b3586 >2013-12-12 12:53:41,482 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 23a45dce >2013-12-12 12:53:42,017 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 23a45dce >2013-12-12 12:53:42,019 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5859aa58 >2013-12-12 12:53:42,206 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 5859aa58 >2013-12-12 12:53:47,213 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 51697419 >2013-12-12 12:53:47,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 51697419 >2013-12-12 12:53:47,546 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6da4ab4d >2013-12-12 12:53:47,728 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6da4ab4d >2013-12-12 12:53:52,734 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6394e9e4 >2013-12-12 12:53:53,298 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6394e9e4 >2013-12-12 12:53:53,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2d991236 >2013-12-12 12:53:53,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 2d991236 >2013-12-12 12:53:58,505 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5de15da1 >2013-12-12 12:53:58,549 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2b546aee >2013-12-12 12:53:58,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5de15da1 >2013-12-12 12:53:59,178 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2b546aee >2013-12-12 12:53:59,180 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1b4cf234 >2013-12-12 12:53:59,445 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 1b4cf234 >2013-12-12 12:54:04,451 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: fbac84d >2013-12-12 12:54:04,961 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: fbac84d >2013-12-12 12:54:04,963 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 51574138 >2013-12-12 12:54:05,172 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 51574138 >2013-12-12 12:54:10,182 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 106564bb >2013-12-12 12:54:10,831 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 106564bb >2013-12-12 12:54:10,833 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 695e8c90 >2013-12-12 12:54:11,058 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 695e8c90 >2013-12-12 12:54:16,080 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6ae0ef8f >2013-12-12 12:54:16,538 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6ae0ef8f >2013-12-12 12:54:16,540 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 42b0f85 >2013-12-12 12:54:16,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 42b0f85 >2013-12-12 12:54:19,918 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [6c77966f] Running command: CreateGlusterVolumeCommand internal: false. Entities affected : ID: 99408929-82cf-4dc7-a532-9d998063fa95 Type: VdsGroups >2013-12-12 12:54:19,919 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [6c77966f] START, CreateGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2d8b1334 >2013-12-12 12:54:21,871 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:24,190 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:24,197 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:24,618 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [6c77966f] FINISH, CreateGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@61854ea5, log id: 2d8b1334 >2013-12-12 12:54:24,751 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [6c77966f] Correlation ID: 6c77966f, Job ID: 9b99f882-3202-442b-9a4a-80d8e7de7ed7, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume boom created. >2013-12-12 12:54:24,801 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-4) [1d20fe07] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 12:54:24,803 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [1d20fe07] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6b146765 >2013-12-12 12:54:27,207 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:27,209 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:29,240 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [1d20fe07] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 6b146765 >2013-12-12 12:54:29,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [1d20fe07] Correlation ID: 1d20fe07, Job ID: 9b99f882-3202-442b-9a4a-80d8e7de7ed7, Call Stack: null, Custom Event ID: -1, Message: Volume Option nfs.disable=off set on boom. >2013-12-12 12:54:29,370 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-4) [30a7e6ea] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 12:54:29,370 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [30a7e6ea] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 35e7fc25 >2013-12-12 12:54:30,423 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:30,429 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:33,439 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:33,444 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:34,291 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [30a7e6ea] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 35e7fc25 >2013-12-12 12:54:34,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [30a7e6ea] Correlation ID: 30a7e6ea, Job ID: 9b99f882-3202-442b-9a4a-80d8e7de7ed7, Call Stack: null, Custom Event ID: -1, Message: Volume Option user.cifs=enable set on boom. >2013-12-12 12:54:34,545 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-4) [4c9fd9d6] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 12:54:34,546 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [4c9fd9d6] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 16fa9079 >2013-12-12 12:54:36,506 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:36,516 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:39,522 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:39,534 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-65) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:39,713 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-4) [4c9fd9d6] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 16fa9079 >2013-12-12 12:54:39,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4c9fd9d6] Correlation ID: 4c9fd9d6, Job ID: 9b99f882-3202-442b-9a4a-80d8e7de7ed7, Call Stack: null, Custom Event ID: -1, Message: Volume Option auth.allow=* set on boom. >2013-12-12 12:54:39,816 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [4c9fd9d6] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:39,820 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 75b2a5c6 >2013-12-12 12:54:40,195 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 75b2a5c6 >2013-12-12 12:54:40,197 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5c2ddb85 >2013-12-12 12:54:40,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2244fc66}, log id: 5c2ddb85 >2013-12-12 12:54:45,443 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 45c7129 >2013-12-12 12:54:45,887 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 45c7129 >2013-12-12 12:54:45,889 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6ae7c617 >2013-12-12 12:54:46,123 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6403f8df}, log id: 6ae7c617 >2013-12-12 12:54:47,233 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-50) [34daee8c] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 12:54:47,235 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-50) [34daee8c] START, StartGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4a71243f >2013-12-12 12:54:47,722 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-50) [34daee8c] FINISH, StartGlusterVolumeVDSCommand, log id: 4a71243f >2013-12-12 12:54:47,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [34daee8c] Correlation ID: 34daee8c, Job ID: ce5e5f2d-7f6a-4b30-8c18-c14ad2208092, Call Stack: null, Custom Event ID: -1, Message: Could not start Gluster Volume boom. >2013-12-12 12:54:47,750 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-50) [34daee8c] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:54:51,139 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 506fe17d >2013-12-12 12:54:51,620 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 506fe17d >2013-12-12 12:54:51,622 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2c2a3aaf >2013-12-12 12:54:51,876 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fe6d0620}, log id: 2c2a3aaf >2013-12-12 12:54:56,907 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 21272c93 >2013-12-12 12:54:57,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 21272c93 >2013-12-12 12:54:57,718 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1d5adfaf >2013-12-12 12:54:57,915 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@82b1c3b8}, log id: 1d5adfaf >2013-12-12 12:54:58,987 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2bc14af3 >2013-12-12 12:54:59,215 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterTasksListVDSCommand, return: [], log id: 2bc14af3 >2013-12-12 12:55:02,948 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 56e2c1f2 >2013-12-12 12:55:03,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 56e2c1f2 >2013-12-12 12:55:03,304 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 57e8fc7c >2013-12-12 12:55:03,671 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@33a675c1}, log id: 57e8fc7c >2013-12-12 12:55:08,700 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4bf7164b >2013-12-12 12:55:09,158 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4bf7164b >2013-12-12 12:55:09,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 21b7ee6b >2013-12-12 12:55:09,377 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@59583564}, log id: 21b7ee6b >2013-12-12 12:55:14,405 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7d4152c9 >2013-12-12 12:55:14,803 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7d4152c9 >2013-12-12 12:55:14,804 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1ec84554 >2013-12-12 12:55:15,007 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5ab2b784}, log id: 1ec84554 >2013-12-12 12:55:15,095 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-50) [3d59bd0b] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 12:55:15,095 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-50) [3d59bd0b] START, StartGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 24548b65 >2013-12-12 12:55:17,915 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-50) [3d59bd0b] FINISH, StartGlusterVolumeVDSCommand, log id: 24548b65 >2013-12-12 12:55:17,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [3d59bd0b] Correlation ID: 3d59bd0b, Job ID: 1aa514c9-4199-413a-80a7-b724f09c7d66, Call Stack: null, Custom Event ID: -1, Message: Could not start Gluster Volume boom. >2013-12-12 12:55:17,945 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-50) [3d59bd0b] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 12:55:20,019 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 92f0cc >2013-12-12 12:55:20,518 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 92f0cc >2013-12-12 12:55:20,520 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: f2ef583 >2013-12-12 12:55:20,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dd74b823}, log id: f2ef583 >2013-12-12 12:55:20,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:55:25,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 352624bc >2013-12-12 12:55:26,311 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 352624bc >2013-12-12 12:55:26,313 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1be78aa6 >2013-12-12 12:55:26,637 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cb6a9231}, log id: 1be78aa6 >2013-12-12 12:55:26,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:55:31,717 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 62a1bbe0 >2013-12-12 12:55:32,073 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 62a1bbe0 >2013-12-12 12:55:32,075 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 945c194 >2013-12-12 12:55:32,449 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ea684156}, log id: 945c194 >2013-12-12 12:55:32,528 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:55:37,534 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4ba227f >2013-12-12 12:55:38,034 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4ba227f >2013-12-12 12:55:38,039 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2e9a83c6 >2013-12-12 12:55:38,325 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@78b8a559}, log id: 2e9a83c6 >2013-12-12 12:55:38,519 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:55:43,522 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: fba26a3 >2013-12-12 12:55:43,922 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: fba26a3 >2013-12-12 12:55:43,924 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 11a89bb8 >2013-12-12 12:55:44,110 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@474db7d3}, log id: 11a89bb8 >2013-12-12 12:55:49,190 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3a332820 >2013-12-12 12:55:49,848 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3a332820 >2013-12-12 12:55:49,849 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 59900f56 >2013-12-12 12:55:50,027 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a3bfed54}, log id: 59900f56 >2013-12-12 12:55:55,042 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 39cf893c >2013-12-12 12:55:55,472 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 39cf893c >2013-12-12 12:55:55,474 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 58606377 >2013-12-12 12:55:55,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a2ce63c4}, log id: 58606377 >2013-12-12 12:55:59,219 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7b1b6d9e >2013-12-12 12:55:59,459 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7b1b6d9e >2013-12-12 12:56:00,716 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4aae969c >2013-12-12 12:56:01,139 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4aae969c >2013-12-12 12:56:01,142 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 769bbf24 >2013-12-12 12:56:01,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7924b5f4}, log id: 769bbf24 >2013-12-12 12:56:01,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:56:06,507 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 37dbd4d6 >2013-12-12 12:56:06,877 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 37dbd4d6 >2013-12-12 12:56:06,881 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 750eb8de >2013-12-12 12:56:07,062 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a28f7663}, log id: 750eb8de >2013-12-12 12:56:07,215 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:56:12,223 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7029166b >2013-12-12 12:56:12,772 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7029166b >2013-12-12 12:56:12,774 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 63175570 >2013-12-12 12:56:13,131 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ab333e6b}, log id: 63175570 >2013-12-12 12:56:13,201 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:56:18,220 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 297d701f >2013-12-12 12:56:18,962 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 297d701f >2013-12-12 12:56:18,964 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 541efdac >2013-12-12 12:56:19,150 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd658505}, log id: 541efdac >2013-12-12 12:56:19,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:56:24,242 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5dbd298f >2013-12-12 12:56:24,734 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5dbd298f >2013-12-12 12:56:24,736 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 16c5b62a >2013-12-12 12:56:24,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@71dc8f01}, log id: 16c5b62a >2013-12-12 12:56:25,109 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:56:30,113 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 414a838b >2013-12-12 12:56:30,578 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 414a838b >2013-12-12 12:56:30,580 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6c8aacc4 >2013-12-12 12:56:30,762 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@78e776f4}, log id: 6c8aacc4 >2013-12-12 12:56:35,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 26a23bb8 >2013-12-12 12:56:36,149 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 26a23bb8 >2013-12-12 12:56:36,151 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 15b72b06 >2013-12-12 12:56:36,373 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f470c4cd}, log id: 15b72b06 >2013-12-12 12:56:36,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:56:41,459 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 17b1d4e8 >2013-12-12 12:56:42,049 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 17b1d4e8 >2013-12-12 12:56:42,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: f508709 >2013-12-12 12:56:42,259 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ef41250c}, log id: f508709 >2013-12-12 12:56:42,344 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:56:47,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 431af7ba >2013-12-12 12:56:47,815 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 431af7ba >2013-12-12 12:56:47,817 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 58d1b1e7 >2013-12-12 12:56:48,009 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ebb21738}, log id: 58d1b1e7 >2013-12-12 12:56:53,019 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4a3af042 >2013-12-12 12:56:53,384 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4a3af042 >2013-12-12 12:56:53,386 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7c267f08 >2013-12-12 12:56:53,674 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e4acb920}, log id: 7c267f08 >2013-12-12 12:56:58,686 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 58e45226 >2013-12-12 12:56:59,201 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 58e45226 >2013-12-12 12:56:59,205 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6b637953 >2013-12-12 12:56:59,463 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2698f1ef >2013-12-12 12:56:59,474 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b2a97cdd}, log id: 6b637953 >2013-12-12 12:56:59,930 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] FINISH, GlusterTasksListVDSCommand, return: [], log id: 2698f1ef >2013-12-12 12:57:04,487 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3045a506 >2013-12-12 12:57:04,892 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3045a506 >2013-12-12 12:57:04,893 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4a7d5514 >2013-12-12 12:57:05,249 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@328ef6eb}, log id: 4a7d5514 >2013-12-12 12:57:10,261 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3eaaa41c >2013-12-12 12:57:10,686 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3eaaa41c >2013-12-12 12:57:10,688 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6466bdbb >2013-12-12 12:57:10,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cf03eda6}, log id: 6466bdbb >2013-12-12 12:57:10,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was stopped, and updated engine DB with it's new status. >2013-12-12 12:57:15,951 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4e620115 >2013-12-12 12:57:16,423 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4e620115 >2013-12-12 12:57:16,425 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 196f38f2 >2013-12-12 12:57:16,637 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@56129e44}, log id: 196f38f2 >2013-12-12 12:57:21,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3b72b942 >2013-12-12 12:57:22,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3b72b942 >2013-12-12 12:57:22,167 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 30e128a8 >2013-12-12 12:57:22,345 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a7b1a38}, log id: 30e128a8 >2013-12-12 12:57:26,624 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Command ListVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:26,644 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:27,354 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2366ffd5 >2013-12-12 12:57:27,740 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2366ffd5 >2013-12-12 12:57:27,741 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3ec1df42 >2013-12-12 12:57:27,918 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3c7bcb2f}, log id: 3ec1df42 >2013-12-12 12:57:28,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume boom of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 12:57:29,652 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:29,653 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:32,659 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:32,662 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:33,007 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6afd028 >2013-12-12 12:57:33,342 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6afd028 >2013-12-12 12:57:33,343 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2ae659d6 >2013-12-12 12:57:33,571 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4cfbb13c}, log id: 2ae659d6 >2013-12-12 12:57:35,757 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-28) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:35,901 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:37,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:37,672 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:38,909 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:38,910 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:42,679 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:42,680 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:43,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:43,918 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:47,690 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:47,691 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:48,926 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:48,928 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:50,696 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:50,697 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:53,701 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:53,702 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:53,935 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:53,936 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 12:57:56,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:56,712 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:57,944 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:57,946 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:59,720 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:57:59,722 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:57:59,933 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-57) No up server in cluster >2013-12-12 12:57:59,933 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-57) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 12:58:00,951 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:00,955 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:02,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:02,727 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:03,961 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:03,965 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:05,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@1d07c716 >2013-12-12 12:58:05,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) HostName = latest-a >2013-12-12 12:58:05,827 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSRecoveringException: Recovering from crash or Initializing >2013-12-12 12:58:06,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSRecoveringException: Recovering from crash or Initializing > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:42) > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:532) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:336) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:235) > at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >, Custom Event ID: -1, Message: Host latest-a is initializing. Message: Recovering from crash or Initializing >2013-12-12 12:58:06,125 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Failed to refresh VDS , vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a, error = Recovering from crash or Initializing, continuing. >2013-12-12 12:58:06,972 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:06,973 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:09,275 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GetHardwareInfoVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, vds=Host[latest-a]), log id: 29c5e2f6 >2013-12-12 12:58:09,315 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GetHardwareInfoVDSCommand, log id: 29c5e2f6 >2013-12-12 12:58:09,371 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: eb49bf1 >2013-12-12 12:58:09,980 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:09,986 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:10,500 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: eb49bf1 >2013-12-12 12:58:10,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host latest-a. >2013-12-12 12:58:10,528 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Cancelling the recovery from crash timer for VDS 81d38815-c70e-4edd-bfbe-9b854d37252d because vds started initializing >2013-12-12 12:58:10,529 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-72) [4ce80175] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 12:58:10,531 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-72) [4ce80175] START, GetGlusterHostUUIDVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6b398042 >2013-12-12 12:58:10,734 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-72) [4ce80175] FINISH, GetGlusterHostUUIDVDSCommand, return: 76102dca-bdfd-448b-a1cc-7a967cf84589, log id: 6b398042 >2013-12-12 12:58:10,754 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-72) [4c0424b6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 12:58:12,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 12:58:12,995 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Failed to refresh VDS , vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 12:58:13,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1fe41707 >2013-12-12 12:58:13,999 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1fe41707 >2013-12-12 12:58:14,001 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7e8a3213 >2013-12-12 12:58:14,205 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9a075b5b}, log id: 7e8a3213 >2013-12-12 12:58:16,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GetHardwareInfoVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, vds=Host[latest-b]), log id: 6bf93027 >2013-12-12 12:58:16,202 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GetHardwareInfoVDSCommand, log id: 6bf93027 >2013-12-12 12:58:16,218 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1a6c527f >2013-12-12 12:58:16,610 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1a6c527f >2013-12-12 12:58:16,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host latest-b. >2013-12-12 12:58:16,623 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-80) [2bb64471] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 12:58:16,625 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-80) [2bb64471] START, GetGlusterHostUUIDVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 490df171 >2013-12-12 12:58:16,831 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-80) [2bb64471] FINISH, GetGlusterHostUUIDVDSCommand, return: 17277992-0ec5-4f41-9f13-3360571eee4e, log id: 490df171 >2013-12-12 12:58:16,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2bb64471] START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5c55e84e >2013-12-12 12:58:17,172 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2bb64471] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5c55e84e >2013-12-12 12:58:17,193 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 12:58:19,218 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 35f73ae5 >2013-12-12 12:58:19,555 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 35f73ae5 >2013-12-12 12:58:19,557 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 786d4a35 >2013-12-12 12:58:19,735 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@429b0d3a}, log id: 786d4a35 >2013-12-12 12:58:24,750 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 49349a7a >2013-12-12 12:58:25,081 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 49349a7a >2013-12-12 12:58:25,083 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 47d1142b >2013-12-12 12:58:25,261 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8ce11449}, log id: 47d1142b >2013-12-12 12:58:30,271 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-97) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 61fc28aa >2013-12-12 12:58:30,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-97) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 61fc28aa >2013-12-12 12:58:30,608 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-97) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 57f8df4f >2013-12-12 12:58:30,785 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-97) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@72a39d0b}, log id: 57f8df4f >2013-12-12 12:58:35,796 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 23763894 >2013-12-12 12:58:36,135 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 23763894 >2013-12-12 12:58:36,136 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2f66adc7 >2013-12-12 12:58:36,313 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ef2c35da}, log id: 2f66adc7 >2013-12-12 12:58:41,327 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3977e8dd >2013-12-12 12:58:41,664 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3977e8dd >2013-12-12 12:58:41,666 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 192f4c54 >2013-12-12 12:58:41,840 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2056397e}, log id: 192f4c54 >2013-12-12 12:58:46,850 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5765ed26 >2013-12-12 12:58:47,186 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5765ed26 >2013-12-12 12:58:47,188 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6d1c5fd7 >2013-12-12 12:58:47,366 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db5497a5}, log id: 6d1c5fd7 >2013-12-12 12:58:52,383 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4a5927be >2013-12-12 12:58:52,718 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4a5927be >2013-12-12 12:58:52,720 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4ea613d3 >2013-12-12 12:58:52,895 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b7da1421}, log id: 4ea613d3 >2013-12-12 12:58:57,917 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4b8af3a3 >2013-12-12 12:58:58,363 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4b8af3a3 >2013-12-12 12:58:58,365 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 436b07f7 >2013-12-12 12:58:58,600 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d2630ad7}, log id: 436b07f7 >2013-12-12 12:58:59,937 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 46dc6899 >2013-12-12 12:59:00,120 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterTasksListVDSCommand, return: [], log id: 46dc6899 >2013-12-12 12:59:03,613 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5b05edc7 >2013-12-12 12:59:03,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5b05edc7 >2013-12-12 12:59:03,951 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 220fc692 >2013-12-12 12:59:04,125 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3b247ab1}, log id: 220fc692 >2013-12-12 12:59:09,134 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1252c1fa >2013-12-12 12:59:09,466 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1252c1fa >2013-12-12 12:59:09,469 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5934ab2e >2013-12-12 12:59:09,645 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d737a6c2}, log id: 5934ab2e >2013-12-12 12:59:14,656 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 58d62a28 >2013-12-12 12:59:15,049 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 58d62a28 >2013-12-12 12:59:15,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 46fc0773 >2013-12-12 12:59:15,296 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6c817f03}, log id: 46fc0773 >2013-12-12 12:59:20,308 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3f978ddd >2013-12-12 12:59:20,645 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3f978ddd >2013-12-12 12:59:20,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 56d181ad >2013-12-12 12:59:20,824 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@37c138a5}, log id: 56d181ad >2013-12-12 12:59:25,836 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 116c625c >2013-12-12 12:59:26,183 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 116c625c >2013-12-12 12:59:26,185 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 465c1c91 >2013-12-12 12:59:26,370 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@18932a3f}, log id: 465c1c91 >2013-12-12 12:59:31,383 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 55b46539 >2013-12-12 12:59:31,715 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 55b46539 >2013-12-12 12:59:31,716 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4076de43 >2013-12-12 12:59:31,891 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@726fa1fe}, log id: 4076de43 >2013-12-12 12:59:36,900 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7d0809c1 >2013-12-12 12:59:37,242 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7d0809c1 >2013-12-12 12:59:37,244 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 61b802d0 >2013-12-12 12:59:37,426 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d0caab40}, log id: 61b802d0 >2013-12-12 12:59:42,439 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 982c02 >2013-12-12 12:59:42,772 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 982c02 >2013-12-12 12:59:42,774 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2e4b8081 >2013-12-12 12:59:42,948 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@38d2520b}, log id: 2e4b8081 >2013-12-12 12:59:47,957 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 738504c7 >2013-12-12 12:59:48,292 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 738504c7 >2013-12-12 12:59:48,293 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 54fe40cc >2013-12-12 12:59:48,469 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@226a62aa}, log id: 54fe40cc >2013-12-12 12:59:53,482 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1c2e24de >2013-12-12 12:59:53,824 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1c2e24de >2013-12-12 12:59:53,826 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1d03911e >2013-12-12 12:59:54,001 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5365fe45}, log id: 1d03911e >2013-12-12 12:59:55,253 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3503e500 >2013-12-12 12:59:55,445 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@534e419, log id: 3503e500 >2013-12-12 12:59:59,010 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1c352f0b >2013-12-12 12:59:59,353 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1c352f0b >2013-12-12 12:59:59,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2b24e922 >2013-12-12 12:59:59,536 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8e983487}, log id: 2b24e922 >2013-12-12 13:00:00,124 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 209cd3c1 >2013-12-12 13:00:00,309 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterTasksListVDSCommand, return: [], log id: 209cd3c1 >2013-12-12 13:00:04,545 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 14146d8 >2013-12-12 13:00:04,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 14146d8 >2013-12-12 13:00:04,880 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3149fa82 >2013-12-12 13:00:05,055 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9da30e5a}, log id: 3149fa82 >2013-12-12 13:00:10,066 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5b37a5fe >2013-12-12 13:00:10,406 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5b37a5fe >2013-12-12 13:00:10,408 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: c4f8416 >2013-12-12 13:00:10,587 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8428dd59, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@de531296}, log id: c4f8416 >2013-12-12 13:00:10,609 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected new volume foo on cluster Default, and added it to engine DB. >2013-12-12 13:00:15,665 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 30fc573e >2013-12-12 13:00:16,128 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 30fc573e >2013-12-12 13:00:16,130 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: e5ca898 >2013-12-12 13:00:16,349 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@56f39563, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@669f7cd5}, log id: e5ca898 >2013-12-12 13:00:16,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Detected that Volume foo of Cluster Default was started, and updated engine DB with it's new status. >2013-12-12 13:00:21,398 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 392bedbb >2013-12-12 13:00:21,742 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 392bedbb >2013-12-12 13:00:21,744 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 50298677 >2013-12-12 13:00:21,923 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c4fa48cb, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f0a361c1}, log id: 50298677 >2013-12-12 13:00:26,936 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1f2b8d61 >2013-12-12 13:00:27,282 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1f2b8d61 >2013-12-12 13:00:27,283 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 280c0a55 >2013-12-12 13:00:27,459 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d34ecbce, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a40940d6}, log id: 280c0a55 >2013-12-12 13:00:32,471 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: fdd1a13 >2013-12-12 13:00:32,809 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: fdd1a13 >2013-12-12 13:00:32,811 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 56a007e5 >2013-12-12 13:00:32,991 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@592388f, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6304187a}, log id: 56a007e5 >2013-12-12 13:00:38,004 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 49666b9b >2013-12-12 13:00:38,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 49666b9b >2013-12-12 13:00:38,343 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 40492254 >2013-12-12 13:00:38,524 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@46a108a8, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f9a30e78}, log id: 40492254 >2013-12-12 13:00:39,435 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-50) [9d512e3] Running command: StopGlusterVolumeCommand internal: false. Entities affected : ID: 91b69da7-27be-4b5f-a00e-6af59f660a80 Type: GlusterVolume >2013-12-12 13:00:39,438 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-50) [9d512e3] START, StopGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2ff8fcfa >2013-12-12 13:00:43,338 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-50) [9d512e3] FINISH, StopGlusterVolumeVDSCommand, log id: 2ff8fcfa >2013-12-12 13:00:43,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [9d512e3] Correlation ID: 9d512e3, Job ID: 5abd8e16-e9d1-4540-a891-bfece3925670, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo stopped. >2013-12-12 13:00:43,377 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-50) [9d512e3] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:00:43,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2ab62e1f >2013-12-12 13:00:43,887 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2ab62e1f >2013-12-12 13:00:43,889 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4e4c750e >2013-12-12 13:00:44,072 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) [2233cd57] FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9e395b7, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a20682f2}, log id: 4e4c750e >2013-12-12 13:00:49,089 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3667f646 >2013-12-12 13:00:49,423 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3667f646 >2013-12-12 13:00:49,424 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7ed0890a >2013-12-12 13:00:49,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@76b805cb, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a47ea0be}, log id: 7ed0890a >2013-12-12 13:00:54,622 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6eadedf0 >2013-12-12 13:00:54,955 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6eadedf0 >2013-12-12 13:00:54,957 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6e0182b3 >2013-12-12 13:00:55,133 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cf379380, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1db56c01}, log id: 6e0182b3 >2013-12-12 13:01:00,146 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2a5a975c >2013-12-12 13:01:00,312 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: a2374e2 >2013-12-12 13:01:00,483 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2a5a975c >2013-12-12 13:01:00,484 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4842b512 >2013-12-12 13:01:00,550 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterTasksListVDSCommand, return: [], log id: a2374e2 >2013-12-12 13:01:00,670 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {91b69da7-27be-4b5f-a00e-6af59f660a80=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@98d3ca29, a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@71ef08dd}, log id: 4842b512 >2013-12-12 13:01:01,585 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-48) [345f0d63] Running command: DeleteGlusterVolumeCommand internal: false. Entities affected : ID: 91b69da7-27be-4b5f-a00e-6af59f660a80 Type: GlusterVolume >2013-12-12 13:01:01,586 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-48) [345f0d63] START, DeleteGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1e4c17b4 >2013-12-12 13:01:01,767 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-48) [345f0d63] FINISH, DeleteGlusterVolumeVDSCommand, log id: 1e4c17b4 >2013-12-12 13:01:01,801 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-48) [345f0d63] Correlation ID: 345f0d63, Job ID: 55cc38c1-ae76-47ad-868d-7f026f6cec7c, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo deleted. >2013-12-12 13:01:01,810 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-48) [345f0d63] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:05,687 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6c7f1219 >2013-12-12 13:01:06,022 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6c7f1219 >2013-12-12 13:01:06,025 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 756671f9 >2013-12-12 13:01:06,201 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ad462048}, log id: 756671f9 >2013-12-12 13:01:09,051 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-50) [6e8924af] Running command: StopGlusterVolumeCommand internal: false. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 13:01:09,052 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-50) [6e8924af] START, StopGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2702a58b >2013-12-12 13:01:11,209 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-20) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:11,994 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-20) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:12,588 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-20) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:15,018 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-20) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:15,196 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-50) [6e8924af] FINISH, StopGlusterVolumeVDSCommand, log id: 2702a58b >2013-12-12 13:01:15,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [6e8924af] Correlation ID: 6e8924af, Job ID: edfb63e0-c848-4d87-ae70-dbb3072005bb, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume boom stopped. >2013-12-12 13:01:15,294 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2821f07a >2013-12-12 13:01:15,294 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-50) [6e8924af] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:15,631 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2821f07a >2013-12-12 13:01:15,632 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1a22f5a7 >2013-12-12 13:01:15,811 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1803f8a7}, log id: 1a22f5a7 >2013-12-12 13:01:16,490 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [cf3857b] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName boom,$vdsGroup Default,ACTION_TYPE_FAILED_GLUSTER_VOLUME_ALREADY_STOPPED,$volumeName boom >2013-12-12 13:01:16,490 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [cf3857b] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:20,820 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6efe234 >2013-12-12 13:01:21,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6efe234 >2013-12-12 13:01:21,166 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: e264f0a >2013-12-12 13:01:21,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@af15395f}, log id: e264f0a >2013-12-12 13:01:26,354 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6044b1ad >2013-12-12 13:01:26,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6044b1ad >2013-12-12 13:01:26,706 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7911814e >2013-12-12 13:01:26,883 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {a28a613a-efe8-4b17-b969-297dd32f9f28=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3ce1ce37}, log id: 7911814e >2013-12-12 13:01:28,326 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-48) [71ff4bcc] Running command: DeleteGlusterVolumeCommand internal: false. Entities affected : ID: a28a613a-efe8-4b17-b969-297dd32f9f28 Type: GlusterVolume >2013-12-12 13:01:28,326 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-48) [71ff4bcc] START, DeleteGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 77200ad4 >2013-12-12 13:01:28,502 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (pool-4-thread-48) [71ff4bcc] FINISH, DeleteGlusterVolumeVDSCommand, log id: 77200ad4 >2013-12-12 13:01:28,518 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-48) [71ff4bcc] Correlation ID: 71ff4bcc, Job ID: 5e044ee2-84ab-4be8-90bd-f04fe5200d8c, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume boom deleted. >2013-12-12 13:01:28,526 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (pool-4-thread-48) [71ff4bcc] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:01:31,893 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1648c3a >2013-12-12 13:01:32,231 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1648c3a >2013-12-12 13:01:32,233 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 52b83e94 >2013-12-12 13:01:32,407 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 52b83e94 >2013-12-12 13:01:37,411 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 266f2c8a >2013-12-12 13:01:37,744 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 266f2c8a >2013-12-12 13:01:37,746 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1fc837eb >2013-12-12 13:01:37,918 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 1fc837eb >2013-12-12 13:01:42,923 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 27deb686 >2013-12-12 13:01:43,256 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 27deb686 >2013-12-12 13:01:43,260 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 455fb3a5 >2013-12-12 13:01:43,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 455fb3a5 >2013-12-12 13:01:48,434 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2961d312 >2013-12-12 13:01:48,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2961d312 >2013-12-12 13:01:48,774 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 35eebd5f >2013-12-12 13:01:48,947 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 35eebd5f >2013-12-12 13:01:53,954 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 72b1476f >2013-12-12 13:01:54,289 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 72b1476f >2013-12-12 13:01:54,290 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: bbfffcb >2013-12-12 13:01:54,464 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: bbfffcb >2013-12-12 13:01:59,473 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 61c83811 >2013-12-12 13:01:59,806 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 61c83811 >2013-12-12 13:01:59,807 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7e4a131a >2013-12-12 13:01:59,978 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7e4a131a >2013-12-12 13:02:00,555 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2e890547 >2013-12-12 13:02:00,738 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterTasksListVDSCommand, return: [], log id: 2e890547 >2013-12-12 13:02:04,985 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 122f7df0 >2013-12-12 13:02:05,323 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 122f7df0 >2013-12-12 13:02:05,325 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 50015bb >2013-12-12 13:02:05,493 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 50015bb >2013-12-12 13:02:10,498 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 54ec3a58 >2013-12-12 13:02:10,835 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 54ec3a58 >2013-12-12 13:02:10,837 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 53b75c3b >2013-12-12 13:02:11,009 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 53b75c3b >2013-12-12 13:02:16,014 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4b7ff5af >2013-12-12 13:02:16,362 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4b7ff5af >2013-12-12 13:02:16,364 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3e1b0886 >2013-12-12 13:02:16,534 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 3e1b0886 >2013-12-12 13:02:21,540 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4f5f43c6 >2013-12-12 13:02:21,876 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4f5f43c6 >2013-12-12 13:02:21,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1bbec04a >2013-12-12 13:02:22,053 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 1bbec04a >2013-12-12 13:02:27,060 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7fd77f89 >2013-12-12 13:02:27,397 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7fd77f89 >2013-12-12 13:02:27,400 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 338f0965 >2013-12-12 13:02:27,581 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 338f0965 >2013-12-12 13:02:32,279 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [35a79a1c] Running command: CreateGlusterVolumeCommand internal: false. Entities affected : ID: 99408929-82cf-4dc7-a532-9d998063fa95 Type: VdsGroups >2013-12-12 13:02:32,280 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [35a79a1c] START, CreateGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5bd6f661 >2013-12-12 13:02:32,585 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:33,605 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:34,149 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:36,332 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [35a79a1c] FINISH, CreateGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9d173af6, log id: 5bd6f661 >2013-12-12 13:02:36,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [35a79a1c] Correlation ID: 35a79a1c, Job ID: a1b73263-ece1-4977-b168-937aacbe07b1, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo created. >2013-12-12 13:02:36,504 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-10) [731f9a34] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:02:36,507 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [731f9a34] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4dbd0b25 >2013-12-12 13:02:36,615 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:37,159 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:39,625 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:40,177 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:40,629 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [731f9a34] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 4dbd0b25 >2013-12-12 13:02:40,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [731f9a34] Correlation ID: 731f9a34, Job ID: a1b73263-ece1-4977-b168-937aacbe07b1, Call Stack: null, Custom Event ID: -1, Message: Volume Option nfs.disable=off set on foo. >2013-12-12 13:02:40,712 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-10) [566aacde] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:02:40,713 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [566aacde] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2d7fcbac >2013-12-12 13:02:42,714 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:43,239 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:44,873 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [566aacde] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 2d7fcbac >2013-12-12 13:02:44,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [566aacde] Correlation ID: 566aacde, Job ID: a1b73263-ece1-4977-b168-937aacbe07b1, Call Stack: null, Custom Event ID: -1, Message: Volume Option user.cifs=enable set on foo. >2013-12-12 13:02:44,946 INFO [org.ovirt.engine.core.bll.gluster.SetGlusterVolumeOptionCommand] (ajp-/127.0.0.1:8702-10) [d362a4e] Running command: SetGlusterVolumeOptionCommand internal: true. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:02:44,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [d362a4e] START, SetGlusterVolumeOptionVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 33630dd8 >2013-12-12 13:02:45,726 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:46,249 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:48,736 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:49,258 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:49,536 INFO [org.ovirt.engine.core.vdsbroker.gluster.SetGlusterVolumeOptionVDSCommand] (ajp-/127.0.0.1:8702-10) [d362a4e] FINISH, SetGlusterVolumeOptionVDSCommand, log id: 33630dd8 >2013-12-12 13:02:49,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [d362a4e] Correlation ID: d362a4e, Job ID: a1b73263-ece1-4977-b168-937aacbe07b1, Call Stack: null, Custom Event ID: -1, Message: Volume Option auth.allow=* set on foo. >2013-12-12 13:02:49,562 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [d362a4e] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:49,564 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1d600c29 >2013-12-12 13:02:49,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1d600c29 >2013-12-12 13:02:49,901 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1340ac5f >2013-12-12 13:02:50,079 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fb7dd4f7}, log id: 1340ac5f >2013-12-12 13:02:54,645 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-48) [38ac17d1] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:02:54,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-48) [38ac17d1] START, StartGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2b1784c7 >2013-12-12 13:02:55,087 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:55,277 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:57,864 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:58,406 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:58,519 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-4) [3e0a4e43] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:58,762 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-48) [38ac17d1] FINISH, StartGlusterVolumeVDSCommand, log id: 2b1784c7 >2013-12-12 13:02:58,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-48) [38ac17d1] Correlation ID: 38ac17d1, Job ID: 1a4abbae-3f52-4a40-98fc-8908fcbba7d2, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo started. >2013-12-12 13:02:58,846 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-4) [3e0a4e43] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:58,846 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-48) [38ac17d1] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:58,846 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7a86d518 >2013-12-12 13:02:59,196 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7a86d518 >2013-12-12 13:02:59,197 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-4) [3e0a4e43] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:02:59,199 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 48eb7fad >2013-12-12 13:02:59,372 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@409a9825}, log id: 48eb7fad >2013-12-12 13:02:59,420 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-40) [3e0a4e43] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:02:59,421 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-40) [3e0a4e43] START, StartGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3d171092 >2013-12-12 13:02:59,603 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-40) [3e0a4e43] FINISH, StartGlusterVolumeVDSCommand, log id: 3d171092 >2013-12-12 13:02:59,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [3e0a4e43] Correlation ID: 3e0a4e43, Job ID: 4ae257ef-68b5-4308-922e-60b0cde1f6ba, Call Stack: null, Custom Event ID: -1, Message: Could not start Gluster Volume foo. >2013-12-12 13:02:59,662 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-40) [3e0a4e43] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:00,742 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 194ec12e >2013-12-12 13:03:00,927 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterTasksListVDSCommand, return: [], log id: 194ec12e >2013-12-12 13:03:04,385 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6554ae49 >2013-12-12 13:03:04,723 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6554ae49 >2013-12-12 13:03:04,725 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1300e583 >2013-12-12 13:03:04,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6ca7daf4}, log id: 1300e583 >2013-12-12 13:03:09,913 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 17ca51c9 >2013-12-12 13:03:10,249 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 17ca51c9 >2013-12-12 13:03:10,252 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 31efb07 >2013-12-12 13:03:10,431 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ed9a4e84}, log id: 31efb07 >2013-12-12 13:03:15,447 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 728bc602 >2013-12-12 13:03:15,784 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 728bc602 >2013-12-12 13:03:15,785 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1a457739 >2013-12-12 13:03:15,963 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ec75472a}, log id: 1a457739 >2013-12-12 13:03:20,971 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 674c07ae >2013-12-12 13:03:21,310 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 674c07ae >2013-12-12 13:03:21,312 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4c402d84 >2013-12-12 13:03:21,487 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@569df0ac}, log id: 4c402d84 >2013-12-12 13:03:26,499 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5000cfd3 >2013-12-12 13:03:26,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5000cfd3 >2013-12-12 13:03:26,837 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 16711368 >2013-12-12 13:03:27,014 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c3cb75fa}, log id: 16711368 >2013-12-12 13:03:30,683 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [48698ce7] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:03:30,685 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [48698ce7] START, AddBricksToGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 59240437 >2013-12-12 13:03:32,024 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:34,188 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:34,712 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:37,107 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [48698ce7] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 59240437 >2013-12-12 13:03:37,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [48698ce7] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:03:37,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [48698ce7] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/asdfsdfasdfsa] on host [latest-b] added to volume [foo] >2013-12-12 13:03:37,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [48698ce7] Correlation ID: 48698ce7, Job ID: bef741d3-9125-4cde-b3d4-0ab2097ee767, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume foo. >2013-12-12 13:03:37,158 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [48698ce7] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:37,159 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 37609ce4 >2013-12-12 13:03:37,504 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 37609ce4 >2013-12-12 13:03:37,506 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 183cc0b9 >2013-12-12 13:03:37,683 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6dbcb1e5}, log id: 183cc0b9 >2013-12-12 13:03:42,695 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 462dfc7 >2013-12-12 13:03:43,032 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 462dfc7 >2013-12-12 13:03:43,036 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3dd1f5ba >2013-12-12 13:03:43,217 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@45bf5ec7}, log id: 3dd1f5ba >2013-12-12 13:03:48,234 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4c14f38a >2013-12-12 13:03:48,569 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4c14f38a >2013-12-12 13:03:48,570 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1213fd78 >2013-12-12 13:03:48,747 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@687dfe09}, log id: 1213fd78 >2013-12-12 13:03:50,009 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [1212cb49] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:03:50,010 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-11) [1212cb49] START, AddBricksToGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7457a459 >2013-12-12 13:03:53,755 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-34) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:55,298 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-34) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:55,820 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-34) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:58,394 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-34) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:58,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-11) [1212cb49] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 7457a459 >2013-12-12 13:03:58,832 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1212cb49] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:03:58,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1212cb49] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/asdfdfasdfasdf] on host [latest-a] added to volume [foo] >2013-12-12 13:03:58,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1212cb49] Correlation ID: 1212cb49, Job ID: ca326ad0-82f2-421b-8d70-3cc6ce63e674, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume foo. >2013-12-12 13:03:58,876 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5839ccde >2013-12-12 13:03:58,876 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [1212cb49] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:03:59,307 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5839ccde >2013-12-12 13:03:59,309 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7f10ade3 >2013-12-12 13:03:59,487 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7d801bf1}, log id: 7f10ade3 >2013-12-12 13:04:00,962 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 85becb4 >2013-12-12 13:04:01,140 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterTasksListVDSCommand, return: [], log id: 85becb4 >2013-12-12 13:04:04,501 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 35b6875d >2013-12-12 13:04:04,832 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 35b6875d >2013-12-12 13:04:04,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2316a625 >2013-12-12 13:04:05,013 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d2716fb7}, log id: 2316a625 >2013-12-12 13:04:10,021 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3c0ae199 >2013-12-12 13:04:10,358 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3c0ae199 >2013-12-12 13:04:10,359 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 17ef4cb7 >2013-12-12 13:04:10,535 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f6de3434}, log id: 17ef4cb7 >2013-12-12 13:04:15,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 73ae905e >2013-12-12 13:04:15,885 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 73ae905e >2013-12-12 13:04:15,889 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4869dd78 >2013-12-12 13:04:16,065 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ad8e4427}, log id: 4869dd78 >2013-12-12 13:04:21,074 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: a01ae29 >2013-12-12 13:04:21,412 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: a01ae29 >2013-12-12 13:04:21,416 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 59e87448 >2013-12-12 13:04:21,593 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cda44286}, log id: 59e87448 >2013-12-12 13:04:26,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 421d8a20 >2013-12-12 13:04:26,939 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 421d8a20 >2013-12-12 13:04:26,940 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 73573a1b >2013-12-12 13:04:27,116 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) [5da2f50a] FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@21be4ef9}, log id: 73573a1b >2013-12-12 13:04:32,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5a08ae0a >2013-12-12 13:04:32,476 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5a08ae0a >2013-12-12 13:04:32,478 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2fe3f0f5 >2013-12-12 13:04:32,656 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@327917da}, log id: 2fe3f0f5 >2013-12-12 13:04:37,664 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6cf77eeb >2013-12-12 13:04:38,003 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6cf77eeb >2013-12-12 13:04:38,005 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 15d29172 >2013-12-12 13:04:38,187 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@879b0e39}, log id: 15d29172 >2013-12-12 13:04:43,197 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 395aa4d3 >2013-12-12 13:04:43,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 395aa4d3 >2013-12-12 13:04:43,537 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: c9ea2c4 >2013-12-12 13:04:43,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1e352496}, log id: c9ea2c4 >2013-12-12 13:04:44,604 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [5a4ce767] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:04:44,604 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) [5a4ce767] START, AddBricksToGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4a82f5e5 >2013-12-12 13:04:48,721 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-10) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:04:49,705 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-10) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:04:50,196 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-10) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:04:52,555 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) [5a4ce767] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 4a82f5e5 >2013-12-12 13:04:52,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [5a4ce767] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:04:52,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [5a4ce767] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/asdfsdfsadfadsfsaf] on host [latest-b] added to volume [foo] >2013-12-12 13:04:52,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [5a4ce767] Correlation ID: 5a4ce767, Job ID: 46c7128a-eca0-4b78-a64b-79c0a3e2d2fc, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume foo. >2013-12-12 13:04:52,647 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1b0b476c >2013-12-12 13:04:52,648 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [5a4ce767] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:04:52,999 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1b0b476c >2013-12-12 13:04:53,003 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 692413e0 >2013-12-12 13:04:53,183 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a79dc652}, log id: 692413e0 >2013-12-12 13:04:55,454 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7218a45b >2013-12-12 13:04:55,641 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@5812029c, log id: 7218a45b >2013-12-12 13:04:58,191 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2dc5062b >2013-12-12 13:04:58,527 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2dc5062b >2013-12-12 13:04:58,531 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 13d5d668 >2013-12-12 13:04:58,709 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5a695e30}, log id: 13d5d668 >2013-12-12 13:05:01,144 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5748b0a9 >2013-12-12 13:05:01,335 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5748b0a9 >2013-12-12 13:05:03,718 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3e8447c5 >2013-12-12 13:05:04,060 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3e8447c5 >2013-12-12 13:05:04,062 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 268bc090 >2013-12-12 13:05:04,239 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a96b8258}, log id: 268bc090 >2013-12-12 13:05:09,249 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7f8abd16 >2013-12-12 13:05:09,585 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7f8abd16 >2013-12-12 13:05:09,587 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5afda174 >2013-12-12 13:05:09,764 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@41bd24e5}, log id: 5afda174 >2013-12-12 13:05:14,775 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 36e3aa38 >2013-12-12 13:05:15,122 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 36e3aa38 >2013-12-12 13:05:15,127 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: faff18b >2013-12-12 13:05:15,306 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cf047b39}, log id: faff18b >2013-12-12 13:05:20,314 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 16aace27 >2013-12-12 13:05:20,651 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 16aace27 >2013-12-12 13:05:20,653 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1b3e6d6c >2013-12-12 13:05:20,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cdd9932f}, log id: 1b3e6d6c >2013-12-12 13:05:25,843 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6e417b62 >2013-12-12 13:05:26,181 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6e417b62 >2013-12-12 13:05:26,183 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1544eaf4 >2013-12-12 13:05:26,362 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@336e0f71}, log id: 1544eaf4 >2013-12-12 13:05:31,371 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 136413a9 >2013-12-12 13:05:31,706 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 136413a9 >2013-12-12 13:05:31,708 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 529e9c39 >2013-12-12 13:05:31,884 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd819522}, log id: 529e9c39 >2013-12-12 13:05:36,897 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 54492457 >2013-12-12 13:05:37,233 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 54492457 >2013-12-12 13:05:37,235 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6154a80 >2013-12-12 13:05:37,412 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@93851387}, log id: 6154a80 >2013-12-12 13:05:42,425 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 51a58007 >2013-12-12 13:05:42,763 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 51a58007 >2013-12-12 13:05:42,765 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1f60443a >2013-12-12 13:05:42,944 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8982d2d9}, log id: 1f60443a >2013-12-12 13:05:47,953 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4289e859 >2013-12-12 13:05:48,288 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4289e859 >2013-12-12 13:05:48,289 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2a39e145 >2013-12-12 13:05:48,466 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6abc12cd}, log id: 2a39e145 >2013-12-12 13:05:53,480 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 58c94e5c >2013-12-12 13:05:53,815 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 58c94e5c >2013-12-12 13:05:53,817 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 61d123e1 >2013-12-12 13:05:53,994 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3378e740}, log id: 61d123e1 >2013-12-12 13:05:59,002 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6f26a70d >2013-12-12 13:05:59,338 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6f26a70d >2013-12-12 13:05:59,340 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4b264124 >2013-12-12 13:05:59,517 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fc6479d5}, log id: 4b264124 >2013-12-12 13:06:01,339 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7a509010 >2013-12-12 13:06:01,437 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [62cd110a] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:06:01,438 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [62cd110a] START, AddBricksToGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 28e66844 >2013-12-12 13:06:01,556 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7a509010 >2013-12-12 13:06:04,527 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:05,327 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:05,642 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:08,336 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:08,652 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:09,068 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [62cd110a] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 28e66844 >2013-12-12 13:06:09,071 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [62cd110a] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:06:09,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [62cd110a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/asdfasdfasdfsdf] on host [latest-a] added to volume [foo] >2013-12-12 13:06:09,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [62cd110a] Correlation ID: 62cd110a, Job ID: a1e90f76-910c-4b06-a7d4-7fe40abfb524, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume foo. >2013-12-12 13:06:09,104 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [62cd110a] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:09,106 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6a924c81 >2013-12-12 13:06:09,573 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6a924c81 >2013-12-12 13:06:09,575 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 347398f4 >2013-12-12 13:06:09,756 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1ced6ad6}, log id: 347398f4 >2013-12-12 13:06:14,767 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2d4bfaa2 >2013-12-12 13:06:15,109 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2d4bfaa2 >2013-12-12 13:06:15,111 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 58c2a2c3 >2013-12-12 13:06:15,289 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@144dfd9d}, log id: 58c2a2c3 >2013-12-12 13:06:17,509 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-5) [1ee60f6e] Lock Acquired to object EngineLock [exclusiveLocks= key: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:17,595 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (pool-4-thread-50) [1ee60f6e] Running command: StartRebalanceGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:06:17,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (pool-4-thread-50) [1ee60f6e] START, StartRebalanceGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 37e20d43 >2013-12-12 13:06:19,583 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [6c0bdd04] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d value: GLUSTER >, sharedLocks= ] >2013-12-12 13:06:19,584 WARN [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [6c0bdd04] CanDoAction of action StartRebalanceGlusterVolume failed. Reasons:VAR__ACTION__REBALANCE_START,VAR__TYPE__GLUSTER_VOLUME,$volumeName foo,$vdsGroup Default,ACTION_TYPE_FAILED_VOLUME_OPERATION_IN_PROGRESS >2013-12-12 13:06:20,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 484cec27 >2013-12-12 13:06:20,640 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 484cec27 >2013-12-12 13:06:20,641 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 604c6e12 >2013-12-12 13:06:20,818 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@971aba7d}, log id: 604c6e12 >2013-12-12 13:06:25,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 38f3d763 >2013-12-12 13:06:28,197 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (pool-4-thread-50) [1ee60f6e] FINISH, StartRebalanceGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@c2190d7, log id: 37e20d43 >2013-12-12 13:06:28,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [1ee60f6e] Correlation ID: 1ee60f6e, Job ID: e81a7671-23af-492f-a8c2-80cef5a86b79, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo rebalance started. >2013-12-12 13:06:28,475 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 38f3d763 >2013-12-12 13:06:28,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2f7244f4 >2013-12-12 13:06:28,713 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@839e9dd3}, log id: 2f7244f4 >2013-12-12 13:06:33,723 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 31fb06c3 >2013-12-12 13:06:34,057 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 31fb06c3 >2013-12-12 13:06:34,059 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 1d3f803f >2013-12-12 13:06:34,238 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@713aff5a}, log id: 1d3f803f >2013-12-12 13:06:39,248 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 45b5f6e1 >2013-12-12 13:06:39,584 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 45b5f6e1 >2013-12-12 13:06:39,585 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4fe344ae >2013-12-12 13:06:39,770 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2961b7d7}, log id: 4fe344ae >2013-12-12 13:06:44,782 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2008de74 >2013-12-12 13:06:45,124 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2008de74 >2013-12-12 13:06:45,126 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 65b01242 >2013-12-12 13:06:45,301 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ff22372e}, log id: 65b01242 >2013-12-12 13:06:50,313 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 72ece427 >2013-12-12 13:06:50,652 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 72ece427 >2013-12-12 13:06:50,653 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 20755a11 >2013-12-12 13:06:50,831 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e2f2e8c}, log id: 20755a11 >2013-12-12 13:06:55,840 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7aa26c31 >2013-12-12 13:06:56,176 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7aa26c31 >2013-12-12 13:06:56,178 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 758f738c >2013-12-12 13:06:56,353 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@69b16a9c}, log id: 758f738c >2013-12-12 13:07:01,362 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 55e1f86b >2013-12-12 13:07:01,561 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterTasksListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2f47f4dd >2013-12-12 13:07:01,604 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-3) [72dc8d5f] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:07:01,707 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 55e1f86b >2013-12-12 13:07:01,709 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 33f42d56 >2013-12-12 13:07:01,710 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ajp-/127.0.0.1:8702-3) [72dc8d5f] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:07:01,894 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1d787225}, log id: 33f42d56 >2013-12-12 13:07:01,900 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [72dc8d5f] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName foo,$vdsGroup Default,ACTION_TYPE_FAILED_GLUSTER_VOLUME_CANNOT_STOP_REBALANCE_IN_PROGRESS >2013-12-12 13:07:01,900 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) [72dc8d5f] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:07:02,006 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterTasksListVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@9b3e28fb], log id: 2f47f4dd >2013-12-12 13:07:02,036 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Rebalance finished on volume foo. >2013-12-12 13:07:06,905 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 70d44a3a >2013-12-12 13:07:07,243 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 70d44a3a >2013-12-12 13:07:07,244 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2da12c3e >2013-12-12 13:07:07,422 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5f9aca8e}, log id: 2da12c3e >2013-12-12 13:07:12,430 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 7bf7a620 >2013-12-12 13:07:12,767 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7bf7a620 >2013-12-12 13:07:12,769 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5ac1f5d2 >2013-12-12 13:07:12,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@537cf83d}, log id: 5ac1f5d2 >2013-12-12 13:07:17,959 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 2042d171 >2013-12-12 13:07:18,294 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2042d171 >2013-12-12 13:07:18,296 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 132a13fe >2013-12-12 13:07:18,473 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@eeca92d}, log id: 132a13fe >2013-12-12 13:07:23,488 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2052f917 >2013-12-12 13:07:23,824 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2052f917 >2013-12-12 13:07:23,826 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 66cbd6e3 >2013-12-12 13:07:24,004 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4cf0a087}, log id: 66cbd6e3 >2013-12-12 13:07:29,014 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1ecd670b >2013-12-12 13:07:29,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1ecd670b >2013-12-12 13:07:29,352 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 363a9850 >2013-12-12 13:07:29,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5a835639}, log id: 363a9850 >2013-12-12 13:07:34,542 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1c6f959d >2013-12-12 13:07:34,882 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1c6f959d >2013-12-12 13:07:34,886 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 3a0ea13d >2013-12-12 13:07:35,064 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@90d74a5}, log id: 3a0ea13d >2013-12-12 13:07:40,074 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 5262789 >2013-12-12 13:07:40,412 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5262789 >2013-12-12 13:07:40,414 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 2cac1a05 >2013-12-12 13:07:40,590 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@23e0474}, log id: 2cac1a05 >2013-12-12 13:07:45,599 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 37a09650 >2013-12-12 13:07:45,936 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 37a09650 >2013-12-12 13:07:45,937 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 1fad42f7 >2013-12-12 13:07:46,115 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@36a8bb7}, log id: 1fad42f7 >2013-12-12 13:07:51,124 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 11789c96 >2013-12-12 13:07:51,464 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 11789c96 >2013-12-12 13:07:51,465 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 15fac3de >2013-12-12 13:07:51,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4b37d2be}, log id: 15fac3de >2013-12-12 13:07:56,654 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 390b0609 >2013-12-12 13:07:56,993 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 390b0609 >2013-12-12 13:07:56,994 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 167e4c5c >2013-12-12 13:07:57,173 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cce7fd0a}, log id: 167e4c5c >2013-12-12 13:08:02,089 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 5bc20078 >2013-12-12 13:08:02,182 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: ab98444 >2013-12-12 13:08:02,768 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterTasksListVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@9b3e28fb], log id: 5bc20078 >2013-12-12 13:08:02,775 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: ab98444 >2013-12-12 13:08:02,776 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 31e3116f >2013-12-12 13:08:02,795 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-68) Trying to release exclusive lock which does not exist, lock key: 8de5e0bb-5151-4c53-b0ea-973b09c27e2dGLUSTER >2013-12-12 13:08:02,954 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9d6de3e7}, log id: 31e3116f >2013-12-12 13:08:07,962 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 43762783 >2013-12-12 13:08:08,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 43762783 >2013-12-12 13:08:08,301 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: a09cfca >2013-12-12 13:08:08,480 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@77c66646}, log id: a09cfca >2013-12-12 13:08:13,489 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: cb0e58e >2013-12-12 13:08:13,825 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: cb0e58e >2013-12-12 13:08:13,827 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 319ffc33 >2013-12-12 13:08:14,003 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@51e6eb9f}, log id: 319ffc33 >2013-12-12 13:08:19,012 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7b831523 >2013-12-12 13:08:19,347 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7b831523 >2013-12-12 13:08:19,349 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 75a00947 >2013-12-12 13:08:19,526 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1bb7c80d}, log id: 75a00947 >2013-12-12 13:08:24,535 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 653a3b40 >2013-12-12 13:08:24,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 653a3b40 >2013-12-12 13:08:24,879 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 71e8af7a >2013-12-12 13:08:25,059 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f2c250b}, log id: 71e8af7a >2013-12-12 13:08:30,068 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 61f4a84b >2013-12-12 13:08:30,416 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 61f4a84b >2013-12-12 13:08:30,417 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 78bfb1d >2013-12-12 13:08:30,594 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e5a95de1}, log id: 78bfb1d >2013-12-12 13:08:35,603 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 215524e8 >2013-12-12 13:08:35,939 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 215524e8 >2013-12-12 13:08:35,941 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 4b233e21 >2013-12-12 13:08:36,118 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fbc4d12b}, log id: 4b233e21 >2013-12-12 13:08:41,126 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 35226d63 >2013-12-12 13:08:41,462 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 35226d63 >2013-12-12 13:08:41,465 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 61a2ed70 >2013-12-12 13:08:41,644 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3ced1111}, log id: 61a2ed70 >2013-12-12 13:08:46,660 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 53a45e07 >2013-12-12 13:08:47,011 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 53a45e07 >2013-12-12 13:08:47,013 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 59260ee >2013-12-12 13:08:47,192 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@30aaa339}, log id: 59260ee >2013-12-12 13:08:52,200 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-41) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 445347ef >2013-12-12 13:08:52,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-41) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 445347ef >2013-12-12 13:08:52,545 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-41) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 482558c3 >2013-12-12 13:08:52,725 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-41) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@41791015}, log id: 482558c3 >2013-12-12 13:08:57,734 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 6c828cae >2013-12-12 13:08:58,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6c828cae >2013-12-12 13:08:58,084 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7c07a2d8 >2013-12-12 13:08:58,260 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1bdbefd2}, log id: 7c07a2d8 >2013-12-12 13:09:02,798 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterTasksListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6e7a8aaf >2013-12-12 13:09:03,150 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterTasksListVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@9b3e28fb], log id: 6e7a8aaf >2013-12-12 13:09:03,177 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-58) Trying to release exclusive lock which does not exist, lock key: 8de5e0bb-5151-4c53-b0ea-973b09c27e2dGLUSTER >2013-12-12 13:09:03,269 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 61f7425 >2013-12-12 13:09:03,611 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 61f7425 >2013-12-12 13:09:03,613 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6b1fbb51 >2013-12-12 13:09:03,790 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d5be2adc}, log id: 6b1fbb51 >2013-12-12 13:09:05,435 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:05,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:05,500 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [66f65670] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:05,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [66f65670] Correlation ID: 66f65670, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:07,233 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:07,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:07,261 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [7fc5a568] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:07,274 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7fc5a568] Correlation ID: 7fc5a568, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:08,639 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:08,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:08,672 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [6d5cd3ba] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:08,683 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [6d5cd3ba] Correlation ID: 6d5cd3ba, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:08,799 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 6a717d56 >2013-12-12 13:09:09,135 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6a717d56 >2013-12-12 13:09:09,137 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3ea79bdd >2013-12-12 13:09:09,315 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@83cece8}, log id: 3ea79bdd >2013-12-12 13:09:09,913 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:09,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:09,948 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [de5a9fc] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:09,957 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [de5a9fc] Correlation ID: de5a9fc, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:11,289 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:11,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:11,924 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [514953b1] Running command: StopGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:09:11,924 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-11) [514953b1] START, StopGlusterVolumeVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 555c8e >2013-12-12 13:09:14,327 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:15,584 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:16,059 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:18,593 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:19,067 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:19,690 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-11) [514953b1] FINISH, StopGlusterVolumeVDSCommand, log id: 555c8e >2013-12-12 13:09:19,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [514953b1] Correlation ID: 514953b1, Job ID: bc1e4adb-7337-4a44-a7ca-2b766bf79a26, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo stopped. >2013-12-12 13:09:19,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 27924963 >2013-12-12 13:09:19,950 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [514953b1] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:19,964 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [3c9cd98b] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:20,036 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [3c9cd98b] Correlation ID: 3c9cd98b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:20,285 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 27924963 >2013-12-12 13:09:20,287 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 3b2f9869 >2013-12-12 13:09:20,465 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {8de5e0bb-5151-4c53-b0ea-973b09c27e2d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f4168e26}, log id: 3b2f9869 >2013-12-12 13:09:21,480 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:21,494 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:21,494 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource] (ajp-/127.0.0.1:8702-10) Could not find sub-resource in the collection resource >2013-12-12 13:09:21,544 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [5a5f91a5] Running command: DeleteGlusterVolumeCommand internal: false. Entities affected : ID: 8de5e0bb-5151-4c53-b0ea-973b09c27e2d Type: GlusterVolume >2013-12-12 13:09:21,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [5a5f91a5] START, DeleteGlusterVolumeVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 33990f >2013-12-12 13:09:21,722 INFO [org.ovirt.engine.core.vdsbroker.gluster.DeleteGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [5a5f91a5] FINISH, DeleteGlusterVolumeVDSCommand, log id: 33990f >2013-12-12 13:09:21,744 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [5a5f91a5] Correlation ID: 5a5f91a5, Job ID: 6ae75d8c-2538-44dd-a8fb-371a2a68abc4, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume foo deleted. >2013-12-12 13:09:21,752 INFO [org.ovirt.engine.core.bll.gluster.DeleteGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [5a5f91a5] Lock freed to object EngineLock [exclusiveLocks= key: 99408929-82cf-4dc7-a532-9d998063fa95 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:09:21,754 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [57bb13b7] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:21,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [57bb13b7] Correlation ID: 57bb13b7, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:22,202 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:22,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:22,244 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [74e01b22] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:22,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [74e01b22] Correlation ID: 74e01b22, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:22,936 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:22,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:22,970 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-6) [557e9114] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 13:09:22,971 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-6) [557e9114] START, SetVdsStatusVDSCommand(HostName = latest-a, HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 5489af02 >2013-12-12 13:09:22,985 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-6) [557e9114] FINISH, SetVdsStatusVDSCommand, log id: 5489af02 >2013-12-12 13:09:23,035 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-6) [557e9114] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 13:09:23,052 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [557e9114] Correlation ID: 557e9114, Job ID: aa99b495-30cf-4280-9216-d666fe842593, Call Stack: null, Custom Event ID: -1, Message: Host latest-a was switched to Maintenance mode by admin@internal. >2013-12-12 13:09:23,098 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [41111098] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:23,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [41111098] Correlation ID: 41111098, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:24,583 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:24,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:24,618 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [30c5a15a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:24,622 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 81d38815-c70e-4edd-bfbe-9b854d37252d : latest-a >2013-12-12 13:09:24,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [30c5a15a] Correlation ID: 30c5a15a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:24,661 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-4-thread-40) Clearing cache of pool: 5849b030-626e-47cb-ad90-3ce782d831b3 for problematic entities of VDS: latest-a. >2013-12-12 13:09:25,479 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 395a2858 >2013-12-12 13:09:25,818 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 395a2858 >2013-12-12 13:09:25,822 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 7b0ad78f >2013-12-12 13:09:25,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7b0ad78f >2013-12-12 13:09:26,824 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:26,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:26,852 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [4cf4acdd] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:26,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [4cf4acdd] Correlation ID: 4cf4acdd, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:27,348 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:27,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:27,398 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] Lock Acquired to object EngineLock [exclusiveLocks= key: 81d38815-c70e-4edd-bfbe-9b854d37252d value: VDS >, sharedLocks= ] >2013-12-12 13:09:27,411 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 81d38815-c70e-4edd-bfbe-9b854d37252d Type: VDS >2013-12-12 13:09:27,414 INFO [org.ovirt.engine.core.vdsbroker.gluster.RemoveGlusterServerVDSCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] START, RemoveGlusterServerVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 4c2374d4 >2013-12-12 13:09:27,590 INFO [org.ovirt.engine.core.vdsbroker.gluster.RemoveGlusterServerVDSCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] FINISH, RemoveGlusterServerVDSCommand, log id: 4c2374d4 >2013-12-12 13:09:27,629 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] START, RemoveVdsVDSCommand( HostId = 81d38815-c70e-4edd-bfbe-9b854d37252d), log id: 56da9747 >2013-12-12 13:09:27,629 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-5) [1611d24c] vdsManager::disposing >2013-12-12 13:09:27,629 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] FINISH, RemoveVdsVDSCommand, log id: 56da9747 >2013-12-12 13:09:27,653 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [1611d24c] Correlation ID: 1611d24c, Call Stack: null, Custom Event ID: -1, Message: Host latest-a was removed by admin@internal. >2013-12-12 13:09:27,653 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-5) [1611d24c] Lock freed to object EngineLock [exclusiveLocks= key: 81d38815-c70e-4edd-bfbe-9b854d37252d value: VDS >, sharedLocks= ] >2013-12-12 13:09:27,654 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [5389e146] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:27,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [5389e146] Correlation ID: 5389e146, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:28,275 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:28,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:28,335 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-11) [622a84a] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 13:09:28,337 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-11) [622a84a] START, SetVdsStatusVDSCommand(HostName = latest-b, HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 37fe2c5e >2013-12-12 13:09:28,344 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (ajp-/127.0.0.1:8702-11) [622a84a] FINISH, SetVdsStatusVDSCommand, log id: 37fe2c5e >2013-12-12 13:09:28,377 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (ajp-/127.0.0.1:8702-11) [622a84a] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 13:09:28,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [622a84a] Correlation ID: 622a84a, Job ID: ee639dd8-ec1f-43a0-9ea9-8bb0eceafbb3, Call Stack: null, Custom Event ID: -1, Message: Host latest-b was switched to Maintenance mode by admin@internal. >2013-12-12 13:09:28,429 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [1d78150e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:28,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1d78150e] Correlation ID: 1d78150e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:30,019 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:30,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:30,138 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [603ae916] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:30,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [603ae916] Correlation ID: 603ae916, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:31,121 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = d43e3091-09a3-4e1b-b07e-8a7272692b99 : latest-b >2013-12-12 13:09:31,145 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-4-thread-40) Clearing cache of pool: 5849b030-626e-47cb-ad90-3ce782d831b3 for problematic entities of VDS: latest-b. >2013-12-12 13:09:31,707 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:31,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:31,737 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [b687284] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:31,744 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [b687284] Correlation ID: b687284, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:32,214 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:32,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:32,246 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [ba5d35a] Lock Acquired to object EngineLock [exclusiveLocks= key: d43e3091-09a3-4e1b-b07e-8a7272692b99 value: VDS >, sharedLocks= ] >2013-12-12 13:09:32,253 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [ba5d35a] Running command: RemoveVdsCommand internal: false. Entities affected : ID: d43e3091-09a3-4e1b-b07e-8a7272692b99 Type: VDS >2013-12-12 13:09:32,278 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [ba5d35a] START, RemoveVdsVDSCommand( HostId = d43e3091-09a3-4e1b-b07e-8a7272692b99), log id: 18dd976 >2013-12-12 13:09:32,278 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [ba5d35a] vdsManager::disposing >2013-12-12 13:09:32,278 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [ba5d35a] FINISH, RemoveVdsVDSCommand, log id: 18dd976 >2013-12-12 13:09:32,285 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [ba5d35a] Correlation ID: ba5d35a, Call Stack: null, Custom Event ID: -1, Message: Host latest-b was removed by admin@internal. >2013-12-12 13:09:32,285 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [ba5d35a] Lock freed to object EngineLock [exclusiveLocks= key: d43e3091-09a3-4e1b-b07e-8a7272692b99 value: VDS >, sharedLocks= ] >2013-12-12 13:09:32,287 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [3005d1e7] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:32,294 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [3005d1e7] Correlation ID: 3005d1e7, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:32,741 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:09:32,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:09:32,775 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [35adfca] Running command: LogoutUserCommand internal: false. >2013-12-12 13:09:32,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [35adfca] Correlation ID: 35adfca, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:09:54,974 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-11) Start refreshing all users data >2013-12-12 13:09:55,190 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2013-12-12 13:10:00,871 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:01,045 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:01,048 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [26216c6c] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:01,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [26216c6c] Correlation ID: 26216c6c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:03,179 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-27) No up server in cluster >2013-12-12 13:10:03,180 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-27) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:10:04,122 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:04,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:04,791 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-1) [6876463c] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:10:04,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [6876463c] Correlation ID: 6876463c, Call Stack: null, Custom Event ID: -1, Message: Host cluster myCluster was added by admin@internal >2013-12-12 13:10:04,859 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [486a69db] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:04,873 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [486a69db] Correlation ID: 486a69db, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:05,445 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:05,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:05,483 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [730c651e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:05,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [730c651e] Correlation ID: 730c651e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:05,990 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:06,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:06,791 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-4) [48b45544] Running command: AddVdsCommand internal: false. Entities affected : ID: 1471ece6-893b-48b6-9531-ad4cec80ad7b Type: VdsGroups >2013-12-12 13:10:06,816 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [44ada428] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:10:06,823 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [44ada428] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 13:10:06,824 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [44ada428] Cannot get vdsManager for vdsid=a89c7427-d95e-46b5-bac9-ce80c2dc7f28 >2013-12-12 13:10:06,825 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] START, RemoveVdsVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6ad61ec6 >2013-12-12 13:10:06,825 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [44ada428] Cannot get vdsManager for vdsid=a89c7427-d95e-46b5-bac9-ce80c2dc7f28 >2013-12-12 13:10:06,825 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] FINISH, RemoveVdsVDSCommand, log id: 6ad61ec6 >2013-12-12 13:10:06,825 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [44ada428] Cannot get vdsManager for vdsid=a89c7427-d95e-46b5-bac9-ce80c2dc7f28 >2013-12-12 13:10:06,825 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] START, AddVdsVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5f90775c >2013-12-12 13:10:06,826 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] AddVds - entered , starting logic to add VDS a89c7427-d95e-46b5-bac9-ce80c2dc7f28 >2013-12-12 13:10:06,826 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] AddVds - VDS a89c7427-d95e-46b5-bac9-ce80c2dc7f28 was added, will try to add it to the resource manager >2013-12-12 13:10:06,826 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [44ada428] Entered VdsManager constructor >2013-12-12 13:10:06,847 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [44ada428] Initialize vdsBroker (latest-a,54,321) >2013-12-12 13:10:06,847 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [44ada428] VDS a89c7427-d95e-46b5-bac9-ce80c2dc7f28 was added to the Resource Manager >2013-12-12 13:10:06,847 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [44ada428] FINISH, AddVdsVDSCommand, log id: 5f90775c >2013-12-12 13:10:06,873 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [44ada428] Correlation ID: 48b45544, Job ID: 59170508-1c46-40d4-8a94-2b287572ffd1, Call Stack: null, Custom Event ID: -1, Message: Host node1 was added by admin@internal. >2013-12-12 13:10:06,880 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [4b934342] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:06,886 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [1c2beaa5] Lock Acquired to object EngineLock [exclusiveLocks= key: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 value: VDS >, sharedLocks= ] >2013-12-12 13:10:06,968 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4b934342] Correlation ID: 4b934342, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:06,991 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [1c2beaa5] Running command: InstallVdsCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:10:06,991 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [1c2beaa5] Before Installation host a89c7427-d95e-46b5-bac9-ce80c2dc7f28, node1 >2013-12-12 13:10:06,992 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [1c2beaa5] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=Installing, nonOperationalReason=NONE), log id: 5cdd0d39 >2013-12-12 13:10:06,998 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [1c2beaa5] FINISH, SetVdsStatusVDSCommand, log id: 5cdd0d39 >2013-12-12 13:10:07,065 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-50) [1c2beaa5] Installation latest-a: Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 13:10:07,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [1c2beaa5] Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 13:10:07,090 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-50) [1c2beaa5] Installation of latest-a. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 13:10:07,094 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-50) SSH execute root@latest-a 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 13:10:07,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Initializing >2013-12-12 13:10:07,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Initializing. >2013-12-12 13:10:07,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment setup >2013-12-12 13:10:07,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment setup. >2013-12-12 13:10:07,684 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment packages setup >2013-12-12 13:10:07,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment packages setup. >2013-12-12 13:10:08,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Programs detection >2013-12-12 13:10:08,386 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Programs detection. >2013-12-12 13:10:08,386 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment customization >2013-12-12 13:10:08,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment customization. >2013-12-12 13:10:08,412 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Logs at host located at: '/tmp/ovirt-host-deploy-20131212131007.log' >2013-12-12 13:10:08,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Logs at host located at: '/tmp/ovirt-host-deploy-20131212131007.log'. >2013-12-12 13:10:08,446 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:08,465 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:08,467 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-a reports unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 >2013-12-12 13:10:08,468 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 to Host latest-a >2013-12-12 13:10:08,472 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [7f666e49] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:08,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7f666e49] Correlation ID: 7f666e49, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:08,632 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Setup validation >2013-12-12 13:10:08,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Setup validation. >2013-12-12 13:10:08,799 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a' >2013-12-12 13:10:08,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Host node1 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a'. >2013-12-12 13:10:08,824 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction setup >2013-12-12 13:10:08,832 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Transaction setup. >2013-12-12 13:10:08,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 13:10:08,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Misc configuration. >2013-12-12 13:10:08,857 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Package installation >2013-12-12 13:10:08,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Package installation. >2013-12-12 13:10:09,848 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Initializing Host: node1 >2013-12-12 13:10:10,424 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 13:10:10,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Misc configuration. >2013-12-12 13:10:10,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Setting time >2013-12-12 13:10:10,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Setting time. >2013-12-12 13:10:10,822 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:10,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:10,855 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [478a8762] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:10,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [478a8762] Correlation ID: 478a8762, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:12,266 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Enrolling certificate >2013-12-12 13:10:12,357 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:12,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Enrolling certificate. >2013-12-12 13:10:12,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:12,414 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [1a641d81] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:12,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [1a641d81] Correlation ID: 1a641d81, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:13,715 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction commit >2013-12-12 13:10:13,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Transaction commit. >2013-12-12 13:10:13,743 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Closing up >2013-12-12 13:10:13,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Closing up. >2013-12-12 13:10:13,757 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting gluster >2013-12-12 13:10:13,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Starting gluster. >2013-12-12 13:10:13,911 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:13,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:13,936 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [438ad072] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:13,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [438ad072] Correlation ID: 438ad072, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:15,473 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:15,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:15,528 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [6578975f] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:15,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [6578975f] Correlation ID: 6578975f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:16,994 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:17,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:17,017 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [44574073] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:17,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [44574073] Correlation ID: 44574073, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:18,573 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:18,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:18,723 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [3b4b85e5] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:18,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [3b4b85e5] Correlation ID: 3b4b85e5, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:19,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting vdsm >2013-12-12 13:10:19,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Starting vdsm. >2013-12-12 13:10:20,217 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:20,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:20,239 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [64f89354] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:20,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [64f89354] Correlation ID: 64f89354, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:21,721 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:21,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:21,766 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [32aba29] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:21,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [32aba29] Correlation ID: 32aba29, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:22,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Pre-termination >2013-12-12 13:10:22,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Pre-termination. >2013-12-12 13:10:23,292 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:23,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:23,323 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [59eea41c] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:23,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [59eea41c] Correlation ID: 59eea41c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:23,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212131023-latest-a-1c2beaa5.log' >2013-12-12 13:10:23,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212131023-latest-a-1c2beaa5.log'. >2013-12-12 13:10:23,934 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Termination >2013-12-12 13:10:23,964 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1c2beaa5, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Termination. >2013-12-12 13:10:24,051 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-50) [1c2beaa5] Engine managed to communicate with VDSM agent on host node1 >2013-12-12 13:10:24,250 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [1c2beaa5] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=Initializing, nonOperationalReason=NONE), log id: 58195244 >2013-12-12 13:10:24,257 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [1c2beaa5] FINISH, SetVdsStatusVDSCommand, log id: 58195244 >2013-12-12 13:10:24,257 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [1c2beaa5] After Installation host node1, VDS >2013-12-12 13:10:24,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [1c2beaa5] Correlation ID: 1c2beaa5, Job ID: 59170508-1c46-40d4-8a94-2b287572ffd1, Call Stack: null, Custom Event ID: -1, Message: Host node1 installed >2013-12-12 13:10:24,274 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [1c2beaa5] Lock freed to object EngineLock [exclusiveLocks= key: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 value: VDS >, sharedLocks= ] >2013-12-12 13:10:24,870 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:24,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:24,906 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [2a12fd9f] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:24,916 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, vds=Host[node1]), log id: 38c9e7bd >2013-12-12 13:10:24,924 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2a12fd9f] Correlation ID: 2a12fd9f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:24,948 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GetHardwareInfoVDSCommand, log id: 38c9e7bd >2013-12-12 13:10:24,967 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 231984ae >2013-12-12 13:10:25,302 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 231984ae >2013-12-12 13:10:25,338 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, vds=Host[node1]), log id: 7475b62d >2013-12-12 13:10:25,345 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GetHardwareInfoVDSCommand, log id: 7475b62d >2013-12-12 13:10:25,359 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 711449a5 >2013-12-12 13:10:25,701 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 711449a5 >2013-12-12 13:10:25,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host node1. >2013-12-12 13:10:25,722 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-44) [1c79c9a3] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:10:25,723 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-44) [1c79c9a3] START, GetGlusterHostUUIDVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6cbd2f07 >2013-12-12 13:10:25,896 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-44) [1c79c9a3] FINISH, GetGlusterHostUUIDVDSCommand, return: 76102dca-bdfd-448b-a1cc-7a967cf84589, log id: 6cbd2f07 >2013-12-12 13:10:25,918 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-44) [66494e88] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:10:26,044 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6d0eb27c >2013-12-12 13:10:26,381 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 6d0eb27c >2013-12-12 13:10:26,383 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 14f4cd45 >2013-12-12 13:10:26,489 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:26,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:26,513 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [4ab0567f] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:26,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [4ab0567f] Correlation ID: 4ab0567f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:26,556 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 14f4cd45 >2013-12-12 13:10:27,024 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:27,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:27,052 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [c7b82d4] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:27,119 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [c7b82d4] Correlation ID: c7b82d4, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:27,586 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:27,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:27,607 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [33232cc9] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:27,618 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [33232cc9] Correlation ID: 33232cc9, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:28,083 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:28,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:28,124 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [38e0d09f] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:28,135 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [38e0d09f] Correlation ID: 38e0d09f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:28,929 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:28,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:29,435 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-1) [354202f7] Running command: AddVdsCommand internal: false. Entities affected : ID: 1471ece6-893b-48b6-9531-ad4cec80ad7b Type: VdsGroups >2013-12-12 13:10:29,453 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:10:29,460 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 13:10:29,460 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-1) [45966fb6] Cannot get vdsManager for vdsid=273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 >2013-12-12 13:10:29,461 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] START, RemoveVdsVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 795e7d68 >2013-12-12 13:10:29,461 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-1) [45966fb6] Cannot get vdsManager for vdsid=273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 >2013-12-12 13:10:29,461 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] FINISH, RemoveVdsVDSCommand, log id: 795e7d68 >2013-12-12 13:10:29,461 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-1) [45966fb6] Cannot get vdsManager for vdsid=273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 >2013-12-12 13:10:29,462 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] START, AddVdsVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 4993ef09 >2013-12-12 13:10:29,462 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] AddVds - entered , starting logic to add VDS 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 >2013-12-12 13:10:29,462 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] AddVds - VDS 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 was added, will try to add it to the resource manager >2013-12-12 13:10:29,462 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-1) [45966fb6] Entered VdsManager constructor >2013-12-12 13:10:29,485 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-1) [45966fb6] Initialize vdsBroker (latest-b,54,321) >2013-12-12 13:10:29,493 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-1) [45966fb6] VDS 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 was added to the Resource Manager >2013-12-12 13:10:29,493 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [45966fb6] FINISH, AddVdsVDSCommand, log id: 4993ef09 >2013-12-12 13:10:29,526 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [45966fb6] Correlation ID: 354202f7, Job ID: d2397a54-86b0-4cc1-8989-0a67fd966087, Call Stack: null, Custom Event ID: -1, Message: Host node2 was added by admin@internal. >2013-12-12 13:10:29,530 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [3da5921e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:29,537 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [6e4381a1] Lock Acquired to object EngineLock [exclusiveLocks= key: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 value: VDS >, sharedLocks= ] >2013-12-12 13:10:29,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [3da5921e] Correlation ID: 3da5921e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:29,560 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [6e4381a1] Running command: InstallVdsCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:10:29,560 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [6e4381a1] Before Installation host 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, node2 >2013-12-12 13:10:29,561 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [6e4381a1] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=Installing, nonOperationalReason=NONE), log id: 7c4c563c >2013-12-12 13:10:29,568 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [6e4381a1] FINISH, SetVdsStatusVDSCommand, log id: 7c4c563c >2013-12-12 13:10:29,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-50) [6e4381a1] Installation latest-b: Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 13:10:29,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [6e4381a1] Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 13:10:29,668 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-50) [6e4381a1] Installation of latest-b. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 13:10:29,677 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-50) SSH execute root@latest-b 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 13:10:30,236 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Initializing >2013-12-12 13:10:30,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Initializing. >2013-12-12 13:10:30,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment setup >2013-12-12 13:10:30,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment setup. >2013-12-12 13:10:30,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment packages setup >2013-12-12 13:10:30,293 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment packages setup. >2013-12-12 13:10:30,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Programs detection >2013-12-12 13:10:30,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Programs detection. >2013-12-12 13:10:30,977 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment customization >2013-12-12 13:10:30,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment customization. >2013-12-12 13:10:30,996 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Logs at host located at: '/tmp/ovirt-host-deploy-20131212131030.log' >2013-12-12 13:10:31,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Logs at host located at: '/tmp/ovirt-host-deploy-20131212131030.log'. >2013-12-12 13:10:31,048 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-b reports unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 >2013-12-12 13:10:31,065 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 to Host latest-b >2013-12-12 13:10:31,123 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:31,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:31,147 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [7c0762e3] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:31,160 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7c0762e3] Correlation ID: 7c0762e3, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:31,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Setup validation >2013-12-12 13:10:31,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Setup validation. >2013-12-12 13:10:31,347 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b' >2013-12-12 13:10:31,360 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Host node2 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b'. >2013-12-12 13:10:31,360 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction setup >2013-12-12 13:10:31,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Transaction setup. >2013-12-12 13:10:31,388 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 13:10:31,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Misc configuration. >2013-12-12 13:10:31,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Package installation >2013-12-12 13:10:31,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Package installation. >2013-12-12 13:10:31,561 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 11a6c9a3 >2013-12-12 13:10:31,896 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 11a6c9a3 >2013-12-12 13:10:31,897 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 2b962132 >2013-12-12 13:10:32,072 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 2b962132 >2013-12-12 13:10:32,495 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Initializing Host: node2 >2013-12-12 13:10:32,674 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:32,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:32,697 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [5aa217f2] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:32,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [5aa217f2] Correlation ID: 5aa217f2, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:32,983 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 13:10:33,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Misc configuration. >2013-12-12 13:10:33,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Setting time >2013-12-12 13:10:33,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Setting time. >2013-12-12 13:10:34,222 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:34,243 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:34,247 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [29f45ba4] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:34,260 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [29f45ba4] Correlation ID: 29f45ba4, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:35,673 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Enrolling certificate >2013-12-12 13:10:35,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Enrolling certificate. >2013-12-12 13:10:35,745 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:35,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:35,857 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [28309f56] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:35,868 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [28309f56] Correlation ID: 28309f56, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:36,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction commit >2013-12-12 13:10:36,968 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Transaction commit. >2013-12-12 13:10:36,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Closing up >2013-12-12 13:10:37,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Closing up. >2013-12-12 13:10:37,002 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting gluster >2013-12-12 13:10:37,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Starting gluster. >2013-12-12 13:10:37,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 764b0962 >2013-12-12 13:10:37,427 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 764b0962 >2013-12-12 13:10:37,428 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6efeee11 >2013-12-12 13:10:37,444 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:37,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:37,477 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [16294129] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:37,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [16294129] Correlation ID: 16294129, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:37,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6efeee11 >2013-12-12 13:10:39,006 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:39,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:39,148 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [3cb96c6e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:39,160 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [3cb96c6e] Correlation ID: 3cb96c6e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:41,704 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:41,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:41,725 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [9e890f2] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:41,738 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [9e890f2] Correlation ID: 9e890f2, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:42,611 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-67) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5e24764b >2013-12-12 13:10:42,952 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 5e24764b >2013-12-12 13:10:42,953 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7355f579 >2013-12-12 13:10:43,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7355f579 >2013-12-12 13:10:43,196 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting vdsm >2013-12-12 13:10:43,238 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Starting vdsm. >2013-12-12 13:10:43,631 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:43,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:43,667 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [6fd9cdb5] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:43,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [6fd9cdb5] Correlation ID: 6fd9cdb5, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:45,866 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Pre-termination >2013-12-12 13:10:45,879 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Pre-termination. >2013-12-12 13:10:46,683 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212131046-latest-b-6e4381a1.log' >2013-12-12 13:10:46,686 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:46,696 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212131046-latest-b-6e4381a1.log'. >2013-12-12 13:10:46,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:46,728 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [3096fd9b] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:46,737 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3096fd9b] Correlation ID: 3096fd9b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:47,277 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Termination >2013-12-12 13:10:47,356 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6e4381a1, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Termination. >2013-12-12 13:10:47,464 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-50) [6e4381a1] Engine managed to communicate with VDSM agent on host node2 >2013-12-12 13:10:47,614 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [6e4381a1] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=Initializing, nonOperationalReason=NONE), log id: 5f49bc53 >2013-12-12 13:10:47,629 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [6e4381a1] FINISH, SetVdsStatusVDSCommand, log id: 5f49bc53 >2013-12-12 13:10:47,629 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [6e4381a1] After Installation host node2, VDS >2013-12-12 13:10:47,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-50) [6e4381a1] Correlation ID: 6e4381a1, Job ID: d2397a54-86b0-4cc1-8989-0a67fd966087, Call Stack: null, Custom Event ID: -1, Message: Host node2 installed >2013-12-12 13:10:47,688 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-50) [6e4381a1] Lock freed to object EngineLock [exclusiveLocks= key: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 value: VDS >, sharedLocks= ] >2013-12-12 13:10:48,136 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 2a917e53 >2013-12-12 13:10:48,482 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 2a917e53 >2013-12-12 13:10:48,483 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: f95113a >2013-12-12 13:10:48,553 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:48,596 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:48,603 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [5a3bb26e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:48,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [5a3bb26e] Correlation ID: 5a3bb26e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:48,654 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: f95113a >2013-12-12 13:10:50,379 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:50,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:50,400 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [13342e09] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:50,413 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [13342e09] Correlation ID: 13342e09, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:50,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, vds=Host[node2]), log id: 7ae80b43 >2013-12-12 13:10:50,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GetHardwareInfoVDSCommand, log id: 7ae80b43 >2013-12-12 13:10:50,589 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 4ae2a901 >2013-12-12 13:10:50,930 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED], log id: 4ae2a901 >2013-12-12 13:10:50,965 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, vds=Host[node2]), log id: 2b2af9c3 >2013-12-12 13:10:50,972 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GetHardwareInfoVDSCommand, log id: 2b2af9c3 >2013-12-12 13:10:51,023 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 63860e5e >2013-12-12 13:10:51,376 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED], log id: 63860e5e >2013-12-12 13:10:51,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host node2. >2013-12-12 13:10:51,394 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:10:51,395 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] START, GetGlusterHostUUIDVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 50e08779 >2013-12-12 13:10:51,566 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] FINISH, GetGlusterHostUUIDVDSCommand, return: 17277992-0ec5-4f41-9f13-3360571eee4e, log id: 50e08779 >2013-12-12 13:10:51,581 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 627f0b9e >2013-12-12 13:10:51,925 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 627f0b9e >2013-12-12 13:10:51,927 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] START, AddGlusterServerVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 3cacffae >2013-12-12 13:10:52,309 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:52,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:52,558 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [a44b2c6] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:52,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [a44b2c6] Correlation ID: a44b2c6, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:53,033 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (DefaultQuartzScheduler_Worker-74) [3741e945] FINISH, AddGlusterServerVDSCommand, log id: 3cacffae >2013-12-12 13:10:53,228 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-74) [4eb80e3b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:10:53,663 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 28c3dda2 >2013-12-12 13:10:54,302 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 28c3dda2 >2013-12-12 13:10:54,305 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 15e3779e >2013-12-12 13:10:54,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 15e3779e >2013-12-12 13:10:54,717 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:54,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:54,769 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [6d6aaf0d] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:54,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [6d6aaf0d] Correlation ID: 6d6aaf0d, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:56,687 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:56,704 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:56,709 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [3d676300] Running command: LogoutUserCommand internal: false. >2013-12-12 13:10:56,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [3d676300] Correlation ID: 3d676300, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:10:57,306 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:10:57,321 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:10:58,118 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [2afefdc8] Running command: CreateGlusterVolumeCommand internal: false. Entities affected : ID: 1471ece6-893b-48b6-9531-ad4cec80ad7b Type: VdsGroups >2013-12-12 13:10:58,119 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [2afefdc8] START, CreateGlusterVolumeVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7cf40242 >2013-12-12 13:10:59,482 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:02,264 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:02,271 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [2afefdc8] FINISH, CreateGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ab6cdc75, log id: 7cf40242 >2013-12-12 13:11:02,356 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:02,365 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [2afefdc8] Correlation ID: 2afefdc8, Job ID: 1604add6-766c-4c93-be7f-d614f1eae0fc, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest created. >2013-12-12 13:11:02,374 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: f775dfa >2013-12-12 13:11:02,374 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [2afefdc8] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:02,389 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [27c3c238] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:02,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [27c3c238] Correlation ID: 27c3c238, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:02,716 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: f775dfa >2013-12-12 13:11:02,717 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 12c44586 >2013-12-12 13:11:02,892 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8fcf92c1}, log id: 12c44586 >2013-12-12 13:11:03,289 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-94) No up server in cluster >2013-12-12 13:11:03,289 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-94) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:11:03,295 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 2176da4f >2013-12-12 13:11:03,349 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:03,373 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:03,379 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [6efb1f6a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:03,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [6efb1f6a] Correlation ID: 6efb1f6a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:03,472 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterTasksListVDSCommand, return: [], log id: 2176da4f >2013-12-12 13:11:03,956 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:03,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:04,023 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [71cd9bd] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:11:04,024 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [71cd9bd] START, StartGlusterVolumeVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 920d17 >2013-12-12 13:11:07,905 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-98) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:08,376 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-98) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:08,410 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-98) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:08,486 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [71cd9bd] FINISH, StartGlusterVolumeVDSCommand, log id: 920d17 >2013-12-12 13:11:08,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [71cd9bd] Correlation ID: 71cd9bd, Job ID: 7758f402-359e-44bd-8e66-89a603aca4d0, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest started. >2013-12-12 13:11:08,640 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 60033b >2013-12-12 13:11:08,641 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [71cd9bd] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:08,644 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [4f8f81a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:08,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [4f8f81a] Correlation ID: 4f8f81a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:08,975 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 60033b >2013-12-12 13:11:08,977 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 21a51043 >2013-12-12 13:11:09,150 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@172b6e2b}, log id: 21a51043 >2013-12-12 13:11:14,159 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 55da026d >2013-12-12 13:11:14,516 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 55da026d >2013-12-12 13:11:14,518 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 7511b8ca >2013-12-12 13:11:14,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@405a02c6}, log id: 7511b8ca >2013-12-12 13:11:15,454 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:15,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:16,081 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-1) [41af3a46] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:11:16,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-1) [41af3a46] START, AddBricksToGlusterVolumeVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 4dc01b27 >2013-12-12 13:11:19,705 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:20,461 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:20,528 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:22,550 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-1) [41af3a46] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 4dc01b27 >2013-12-12 13:11:22,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [41af3a46] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:11:22,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [41af3a46] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212131115583927299489] on host [latest-b] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:11:22,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [41af3a46] Correlation ID: 41af3a46, Job ID: b987ad7e-6cf0-4548-b23d-9908cf52b55a, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:11:22,581 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-1) [41af3a46] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:22,583 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 25c970bb >2013-12-12 13:11:22,593 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [22f19e96] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:22,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [22f19e96] Correlation ID: 22f19e96, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:22,928 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 25c970bb >2013-12-12 13:11:22,929 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 64df3a35 >2013-12-12 13:11:23,104 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) [21d4fcb4] FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@82131cae}, log id: 64df3a35 >2013-12-12 13:11:23,411 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:23,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:23,490 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [7e294ddb] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:11:23,490 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [7e294ddb] START, AddBricksToGlusterVolumeVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 42acd64e >2013-12-12 13:11:28,114 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-23) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:29,526 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-23) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:29,557 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-23) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:30,416 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [7e294ddb] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 42acd64e >2013-12-12 13:11:30,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7e294ddb] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:11:30,418 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7e294ddb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212131123545341374234] on host [latest-a] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:11:30,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7e294ddb] Correlation ID: 7e294ddb, Job ID: 0f799f57-5a8f-4700-b298-dc34ff77eeae, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:11:30,449 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6658e20 >2013-12-12 13:11:30,449 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [7e294ddb] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:30,454 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [7bf5755d] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:30,465 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7bf5755d] Correlation ID: 7bf5755d, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:30,797 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6658e20 >2013-12-12 13:11:30,799 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 34e12a2b >2013-12-12 13:11:30,978 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a953c792}, log id: 34e12a2b >2013-12-12 13:11:31,018 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:31,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:31,106 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [1c521b9] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:11:31,107 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [1c521b9] START, AddBricksToGlusterVolumeVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 5fa3d6be >2013-12-12 13:11:36,020 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-38) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:38,364 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) [1c521b9] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 5fa3d6be >2013-12-12 13:11:38,365 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1c521b9] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:11:38,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1c521b9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212131131033219930446] on host [latest-b] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:11:38,461 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1c521b9] Correlation ID: 1c521b9, Job ID: cdd7bd8b-12c5-46e8-a2df-5bbbfe0a3ed0, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:11:38,479 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7e841a5a >2013-12-12 13:11:38,480 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [1c521b9] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:38,490 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [3158c114] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:38,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [3158c114] Correlation ID: 3158c114, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:38,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7e841a5a >2013-12-12 13:11:38,935 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 56d02f2d >2013-12-12 13:11:39,114 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1c83b47}, log id: 56d02f2d >2013-12-12 13:11:39,193 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:11:39,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:11:39,341 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [29a7d9a8] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:11:39,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [29a7d9a8] START, AddBricksToGlusterVolumeVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 163c5dc5 >2013-12-12 13:11:44,123 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-44) [66494e88] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:44,693 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-44) [66494e88] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:44,694 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-44) [66494e88] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:47,215 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [29a7d9a8] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 163c5dc5 >2013-12-12 13:11:47,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [29a7d9a8] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:11:47,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [29a7d9a8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/201312121311391031151947411] on host [latest-a] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:11:47,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [29a7d9a8] Correlation ID: 29a7d9a8, Job ID: ddaf3c55-c1f8-41cb-a6ee-4798e4c02464, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:11:47,287 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) [66494e88] START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5ed4bdbd >2013-12-12 13:11:47,287 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [29a7d9a8] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:11:47,295 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [2be90861] Running command: LogoutUserCommand internal: false. >2013-12-12 13:11:47,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2be90861] Correlation ID: 2be90861, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:11:47,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) [66494e88] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5ed4bdbd >2013-12-12 13:11:47,732 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) [66494e88] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5603903c >2013-12-12 13:11:47,909 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) [66494e88] FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f9813471}, log id: 5603903c >2013-12-12 13:11:52,922 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 33c9a15d >2013-12-12 13:11:53,253 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 33c9a15d >2013-12-12 13:11:53,255 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 526cd12 >2013-12-12 13:11:53,431 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bebf17ad}, log id: 526cd12 >2013-12-12 13:11:58,441 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 1d5cb08d >2013-12-12 13:11:58,782 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1d5cb08d >2013-12-12 13:11:58,784 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 30b992ea >2013-12-12 13:11:58,968 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a1cab799}, log id: 30b992ea >2013-12-12 13:12:03,475 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-68) No up server in cluster >2013-12-12 13:12:03,476 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-68) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:12:03,482 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 648f2999 >2013-12-12 13:12:03,663 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterTasksListVDSCommand, return: [], log id: 648f2999 >2013-12-12 13:12:03,982 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-76) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 33ef14fa >2013-12-12 13:12:04,322 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 33ef14fa >2013-12-12 13:12:04,323 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-76) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 31f008c4 >2013-12-12 13:12:04,499 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@592eed67}, log id: 31f008c4 >2013-12-12 13:12:09,511 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 58d980c6 >2013-12-12 13:12:09,847 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 58d980c6 >2013-12-12 13:12:09,848 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7b52e2ef >2013-12-12 13:12:10,025 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e9034ff3}, log id: 7b52e2ef >2013-12-12 13:12:15,035 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 49984c27 >2013-12-12 13:12:15,366 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 49984c27 >2013-12-12 13:12:15,368 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 32c56822 >2013-12-12 13:12:15,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a87d80a6}, log id: 32c56822 >2013-12-12 13:12:20,554 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 6a2c7018 >2013-12-12 13:12:20,887 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6a2c7018 >2013-12-12 13:12:20,889 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 7bbb73ef >2013-12-12 13:12:21,074 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db3db9c9}, log id: 7bbb73ef >2013-12-12 13:12:26,087 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 26bd7df6 >2013-12-12 13:12:26,422 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 26bd7df6 >2013-12-12 13:12:26,424 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 4cc48a2f >2013-12-12 13:12:26,599 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@eeb99512}, log id: 4cc48a2f >2013-12-12 13:12:31,609 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: eb13734 >2013-12-12 13:12:31,942 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: eb13734 >2013-12-12 13:12:31,946 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 4735ff83 >2013-12-12 13:12:32,121 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c85700ae}, log id: 4735ff83 >2013-12-12 13:12:37,131 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 3e2b5535 >2013-12-12 13:12:37,463 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3e2b5535 >2013-12-12 13:12:37,464 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 536d67c4 >2013-12-12 13:12:37,639 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@abfc71a9}, log id: 536d67c4 >2013-12-12 13:12:42,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6cbef94d >2013-12-12 13:12:42,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6cbef94d >2013-12-12 13:12:42,986 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 174f1446 >2013-12-12 13:12:43,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@775f0305}, log id: 174f1446 >2013-12-12 13:12:48,170 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: e1bc0e0 >2013-12-12 13:12:48,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: e1bc0e0 >2013-12-12 13:12:48,510 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 4e076bb8 >2013-12-12 13:12:48,689 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@975564f2}, log id: 4e076bb8 >2013-12-12 13:12:53,699 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 64f9fd4b >2013-12-12 13:12:54,035 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 64f9fd4b >2013-12-12 13:12:54,037 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 469428bc >2013-12-12 13:12:54,215 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b77cdadd}, log id: 469428bc >2013-12-12 13:12:59,293 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 37c67bb3 >2013-12-12 13:12:59,633 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 37c67bb3 >2013-12-12 13:12:59,635 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 8700006 >2013-12-12 13:12:59,817 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@867074b1}, log id: 8700006 >2013-12-12 13:13:03,671 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-49) No up server in cluster >2013-12-12 13:13:03,671 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-49) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:13:03,673 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 66312b7c >2013-12-12 13:13:03,854 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterTasksListVDSCommand, return: [], log id: 66312b7c >2013-12-12 13:13:04,827 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 27439e34 >2013-12-12 13:13:05,163 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 27439e34 >2013-12-12 13:13:05,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 7d292ad >2013-12-12 13:13:05,344 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e5aca917}, log id: 7d292ad >2013-12-12 13:13:10,354 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-67) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: c3e6b61 >2013-12-12 13:13:10,682 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: c3e6b61 >2013-12-12 13:13:10,684 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 15098864 >2013-12-12 13:13:10,861 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@602e6ce4}, log id: 15098864 >2013-12-12 13:13:15,870 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 2a55a9f8 >2013-12-12 13:13:16,209 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2a55a9f8 >2013-12-12 13:13:16,210 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 64dde2ca >2013-12-12 13:13:16,386 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d5431317}, log id: 64dde2ca >2013-12-12 13:13:21,396 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: db1b2b1 >2013-12-12 13:13:21,728 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: db1b2b1 >2013-12-12 13:13:21,729 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 725c850 >2013-12-12 13:13:21,904 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@87388c19}, log id: 725c850 >2013-12-12 13:13:26,914 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 261388c5 >2013-12-12 13:13:27,251 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 261388c5 >2013-12-12 13:13:27,253 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 16ce2ab7 >2013-12-12 13:13:27,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@422080cf}, log id: 16ce2ab7 >2013-12-12 13:13:32,440 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5fb8729e >2013-12-12 13:13:32,776 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5fb8729e >2013-12-12 13:13:32,778 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 64f5139f >2013-12-12 13:13:32,953 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@75f2d122}, log id: 64f5139f >2013-12-12 13:13:37,965 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7db4f31e >2013-12-12 13:13:38,304 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7db4f31e >2013-12-12 13:13:38,305 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 49f0e61d >2013-12-12 13:13:38,488 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9470ecb4}, log id: 49f0e61d >2013-12-12 13:13:43,503 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 4b27b562 >2013-12-12 13:13:43,840 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4b27b562 >2013-12-12 13:13:43,842 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 40ce4e82 >2013-12-12 13:13:44,021 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4b0c67be}, log id: 40ce4e82 >2013-12-12 13:13:45,823 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-40) [40feb2af] Running command: StopGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:13:45,823 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-40) [40feb2af] START, StopGlusterVolumeVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 498c2113 >2013-12-12 13:13:49,030 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:13:51,643 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:13:51,734 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:13:53,943 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-40) [40feb2af] FINISH, StopGlusterVolumeVDSCommand, log id: 498c2113 >2013-12-12 13:13:54,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [40feb2af] Correlation ID: 40feb2af, Job ID: b132c765-3607-407c-848f-17ae53afb581, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest stopped. >2013-12-12 13:13:54,074 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-40) [40feb2af] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:13:54,075 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 32b6a0ff >2013-12-12 13:13:54,408 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 32b6a0ff >2013-12-12 13:13:54,409 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 77352e12 >2013-12-12 13:13:54,584 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@97c3a7b8}, log id: 77352e12 >2013-12-12 13:13:59,593 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 67e583e6 >2013-12-12 13:13:59,931 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 67e583e6 >2013-12-12 13:13:59,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 208cc65d >2013-12-12 13:14:00,111 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@16882d7e}, log id: 208cc65d >2013-12-12 13:14:01,916 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-40) [48abc2cc] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:14:01,916 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-40) [48abc2cc] START, StartGlusterVolumeVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 4e470ea6 >2013-12-12 13:14:03,857 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-29) No up server in cluster >2013-12-12 13:14:03,857 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-29) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:14:03,859 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 34021329 >2013-12-12 13:14:04,037 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterTasksListVDSCommand, log id: 34021329 >2013-12-12 13:14:04,037 ERROR [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-29) org.ovirt.engine.core.common.errors.VDSError@f149149 >2013-12-12 13:14:04,037 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-29) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: Command execution failed >return code: 2 (Failed with error GlusterVolumeStatusAllFailedException and code 4161) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:14:05,121 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-39) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:06,767 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-39) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:06,835 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-39) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:07,693 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-40) [48abc2cc] FINISH, StartGlusterVolumeVDSCommand, log id: 4e470ea6 >2013-12-12 13:14:07,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-40) [48abc2cc] Correlation ID: 48abc2cc, Job ID: 9fc1e2f9-6021-47e6-a001-404eda413028, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest started. >2013-12-12 13:14:07,808 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 615cbc52 >2013-12-12 13:14:07,809 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-40) [48abc2cc] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:08,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 615cbc52 >2013-12-12 13:14:08,164 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 595ee5d5 >2013-12-12 13:14:08,340 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1130d68b}, log id: 595ee5d5 >2013-12-12 13:14:13,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 44dfc044 >2013-12-12 13:14:13,694 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 44dfc044 >2013-12-12 13:14:13,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 2e5fec69 >2013-12-12 13:14:13,871 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ae84f1b8}, log id: 2e5fec69 >2013-12-12 13:14:14,366 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:14,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:14,410 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [74b8d5e9] Lock Acquired to object EngineLock [exclusiveLocks= key: 710d037b-f0d7-4460-b13c-cea331ee8ac9 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:14,423 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [74b8d5e9] Running command: StartRebalanceGlusterVolumeCommand internal: false. Entities affected : ID: 710d037b-f0d7-4460-b13c-cea331ee8ac9 Type: GlusterVolume >2013-12-12 13:14:14,433 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [74b8d5e9] START, StartRebalanceGlusterVolumeVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 29dfffa8 >2013-12-12 13:14:18,881 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 22cb49b5 >2013-12-12 13:14:19,217 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 22cb49b5 >2013-12-12 13:14:19,219 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 6ce8b9d0 >2013-12-12 13:14:19,395 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {710d037b-f0d7-4460-b13c-cea331ee8ac9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c91219ac}, log id: 6ce8b9d0 >2013-12-12 13:14:19,892 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [74b8d5e9] FINISH, StartRebalanceGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@67e12cdf, log id: 29dfffa8 >2013-12-12 13:14:19,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [74b8d5e9] Correlation ID: 74b8d5e9, Job ID: 3f9682da-e426-418f-b80a-75e54fe0a2c8, Call Stack: null, Custom Event ID: -1, Message: Could not start Gluster Volume StartMigrationDuringRebalanceTest rebalance. >2013-12-12 13:14:20,017 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [74b8d5e9] Lock freed to object EngineLock [exclusiveLocks= key: 710d037b-f0d7-4460-b13c-cea331ee8ac9 value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:20,022 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-7) Operation Failed: [command execution failed >error: connection failed. please check if gluster daemon is operational. >return code: 1] >2013-12-12 13:14:20,024 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [7275928d] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:20,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7275928d] Correlation ID: 7275928d, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:20,476 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:20,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:20,527 WARN [org.ovirt.engine.core.bll.gluster.StopRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [2396ea3c] CanDoAction of action StopRebalanceGlusterVolume failed. Reasons:VAR__ACTION__REBALANCE_STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,ACTION_TYPE_FAILED_GLUSTER_VOLUME_REBALANCE_NOT_STARTED >2013-12-12 13:14:20,531 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-4) Operation Failed: [Cannot stop rebalance Gluster Volume. Rebalance is not running on the volume StartMigrationDuringRebalanceTest in cluster myCluster.] >2013-12-12 13:14:20,533 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [2ae61180] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:20,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [2ae61180] Correlation ID: 2ae61180, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:24,406 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 221e29d2 >2013-12-12 13:14:27,433 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, log id: 221e29d2 >2013-12-12 13:14:27,434 INFO [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-59) Gluster servers list failed in server node2 moving it to NonOperational >2013-12-12 13:14:27,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster peer list from server node2 on Cluster myCluster. >2013-12-12 13:14:27,482 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [6b45cac2] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:14:27,483 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [6b45cac2] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 1958530b >2013-12-12 13:14:27,490 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [6b45cac2] FINISH, SetVdsStatusVDSCommand, log id: 1958530b >2013-12-12 13:14:27,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [6b45cac2] Correlation ID: 6b45cac2, Job ID: fe95f8ab-311a-439b-acd1-6e114798c241, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node2. >2013-12-12 13:14:27,541 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) [6b45cac2] START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 3fb544c7 >2013-12-12 13:14:30,562 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) [6b45cac2] FINISH, GlusterServersListVDSCommand, log id: 3fb544c7 >2013-12-12 13:14:30,562 INFO [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-59) [6b45cac2] Gluster servers list failed in server node1 moving it to NonOperational >2013-12-12 13:14:30,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [6b45cac2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster peer list from server node1 on Cluster myCluster. >2013-12-12 13:14:30,624 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [2b3d0564] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:14:30,625 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [2b3d0564] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 5e8313bd >2013-12-12 13:14:30,632 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [2b3d0564] FINISH, SetVdsStatusVDSCommand, log id: 5e8313bd >2013-12-12 13:14:30,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [2b3d0564] Correlation ID: 2b3d0564, Job ID: fd9b7e80-efd6-4e26-8174-f757d07c0476, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node1. >2013-12-12 13:14:30,657 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-59) [2b3d0564] gluster peer status command failed on all servers of the cluster myCluster.Cant refresh its data at this point. >2013-12-12 13:14:30,658 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) [2b3d0564] START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 36c28b1b >2013-12-12 13:14:33,673 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) [2b3d0564] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 36c28b1b >2013-12-12 13:14:33,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [2b3d0564] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster volume list from server node2. >2013-12-12 13:14:33,723 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [30bb3c11] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:14:33,725 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [30bb3c11] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 331b6e16 >2013-12-12 13:14:33,732 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [30bb3c11] FINISH, SetVdsStatusVDSCommand, log id: 331b6e16 >2013-12-12 13:14:33,748 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [30bb3c11] Correlation ID: 30bb3c11, Job ID: 8006621d-a635-4371-a478-5726972dc52f, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node2. >2013-12-12 13:14:33,757 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) [30bb3c11] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 7bfe52de >2013-12-12 13:14:34,901 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:34,923 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:34,956 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [3936aa76] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:34,965 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [3936aa76] Correlation ID: 3936aa76, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:36,772 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) [30bb3c11] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7bfe52de >2013-12-12 13:14:36,846 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [30bb3c11] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster volume list from server node1. >2013-12-12 13:14:36,892 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [1a8dbadf] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:14:36,894 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [1a8dbadf] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 265f2388 >2013-12-12 13:14:36,901 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [1a8dbadf] FINISH, SetVdsStatusVDSCommand, log id: 265f2388 >2013-12-12 13:14:36,918 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [1a8dbadf] Correlation ID: 1a8dbadf, Job ID: 4391da02-bccf-4860-a932-3f2d812100bc, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node1. >2013-12-12 13:14:36,926 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-59) [1a8dbadf] gluster volume info command failed on all servers of the cluster myCluster.Cant refresh its data at this point. >2013-12-12 13:14:41,420 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:41,435 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:41,457 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [378d1492] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:41,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [378d1492] Correlation ID: 378d1492, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:43,210 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:43,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:43,260 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [445cb5e3] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,$clusterName myCluster,ACTION_TYPE_FAILED_NO_UP_SERVER_FOUND >2013-12-12 13:14:43,260 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [445cb5e3] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:43,277 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-7) Operation Failed: [Cannot stop Gluster Volume. No up server found in myCluster.] >2013-12-12 13:14:43,278 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [b2c0931] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:43,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [b2c0931] Correlation ID: b2c0931, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:44,739 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:44,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:44,787 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-9) [3de7163b] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,$clusterName myCluster,ACTION_TYPE_FAILED_NO_UP_SERVER_FOUND >2013-12-12 13:14:44,787 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-9) [3de7163b] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:44,802 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-9) Operation Failed: [Cannot stop Gluster Volume. No up server found in myCluster.] >2013-12-12 13:14:44,808 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [5f52f1c6] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:44,819 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [5f52f1c6] Correlation ID: 5f52f1c6, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:14:46,255 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:14:46,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:14:46,345 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [1fc8cad6] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,$clusterName myCluster,ACTION_TYPE_FAILED_NO_UP_SERVER_FOUND >2013-12-12 13:14:46,345 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [1fc8cad6] Lock freed to object EngineLock [exclusiveLocks= key: 1471ece6-893b-48b6-9531-ad4cec80ad7b value: GLUSTER >, sharedLocks= ] >2013-12-12 13:14:46,361 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-7) Operation Failed: [Cannot stop Gluster Volume. No up server found in myCluster.] >2013-12-12 13:14:46,361 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [409fc30e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:14:46,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [409fc30e] Correlation ID: 409fc30e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:15:00,012 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-10) Autorecovering 2 hosts >2013-12-12 13:15:00,012 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-10) Autorecovering hosts id: a89c7427-d95e-46b5-bac9-ce80c2dc7f28, name : node1 >2013-12-12 13:15:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] Lock Acquired to object EngineLock [exclusiveLocks= key: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 value: VDS >, sharedLocks= ] >2013-12-12 13:15:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] Running command: ActivateVdsCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:15:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] Before acquiring lock in order to prevent monitoring for host node1 from data-center Default >2013-12-12 13:15:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] Lock acquired, from now a monitoring of host will be skipped for host node1 from data-center Default >2013-12-12 13:15:00,019 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=Unassigned, nonOperationalReason=NONE), log id: 1bcd70e9 >2013-12-12 13:15:00,035 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] FINISH, SetVdsStatusVDSCommand, log id: 1bcd70e9 >2013-12-12 13:15:00,037 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] START, ActivateVdsVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 1fb53c59 >2013-12-12 13:15:00,072 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, vds=Host[node1]), log id: 343fa6e5 >2013-12-12 13:15:00,081 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] FINISH, GetHardwareInfoVDSCommand, log id: 343fa6e5 >2013-12-12 13:15:00,095 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] START, GlusterServersListVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28), log id: 5929c132 >2013-12-12 13:15:03,111 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) [6aa06386] FINISH, GlusterServersListVDSCommand, log id: 5929c132 >2013-12-12 13:15:03,227 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-10) [44b4d680] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:15:03,228 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [44b4d680] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a89c7427-d95e-46b5-bac9-ce80c2dc7f28, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 75dc65fb >2013-12-12 13:15:03,304 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [44b4d680] FINISH, SetVdsStatusVDSCommand, log id: 75dc65fb >2013-12-12 13:15:03,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [44b4d680] Correlation ID: 44b4d680, Job ID: a2c0825a-5f94-4ac1-8945-7c0e97fadac9, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node1. >2013-12-12 13:15:03,426 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 Type: VDS >2013-12-12 13:15:03,427 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] FINISH, ActivateVdsVDSCommand, return: Host[node1], log id: 1fb53c59 >2013-12-12 13:15:03,450 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] Activate finished. Lock released. Monitoring can run now for host node1 from data-center Default >2013-12-12 13:15:03,458 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] Correlation ID: 6aa06386, Call Stack: null, Custom Event ID: -1, Message: Host node1 was autorecovered. >2013-12-12 13:15:03,458 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] Lock freed to object EngineLock [exclusiveLocks= key: a89c7427-d95e-46b5-bac9-ce80c2dc7f28 value: VDS >, sharedLocks= ] >2013-12-12 13:15:03,458 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-10) [3bf7aef4] Autorecovering hosts id: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, name : node2 >2013-12-12 13:15:03,459 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [10894266] Lock Acquired to object EngineLock [exclusiveLocks= key: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 value: VDS >, sharedLocks= ] >2013-12-12 13:15:03,459 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [10894266] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:15:03,460 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [10894266] Before acquiring lock in order to prevent monitoring for host node2 from data-center Default >2013-12-12 13:15:03,460 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [10894266] Lock acquired, from now a monitoring of host will be skipped for host node2 from data-center Default >2013-12-12 13:15:03,468 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=Unassigned, nonOperationalReason=NONE), log id: 1a872ce4 >2013-12-12 13:15:03,474 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] FINISH, SetVdsStatusVDSCommand, log id: 1a872ce4 >2013-12-12 13:15:03,483 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] START, ActivateVdsVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 6e37ba65 >2013-12-12 13:15:03,521 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, vds=Host[node2]), log id: bb56e4c >2013-12-12 13:15:03,528 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] FINISH, GetHardwareInfoVDSCommand, log id: bb56e4c >2013-12-12 13:15:03,545 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] START, GlusterServersListVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69), log id: 21d3f6c1 >2013-12-12 13:15:04,040 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-8) No up server in cluster >2013-12-12 13:15:04,040 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-8) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:15:04,041 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-8) No up server in cluster >2013-12-12 13:15:04,041 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-8) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:15:06,559 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) [10894266] FINISH, GlusterServersListVDSCommand, log id: 21d3f6c1 >2013-12-12 13:15:06,608 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-10) [59bdcda] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:15:06,609 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [59bdcda] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 64d54052 >2013-12-12 13:15:06,616 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [59bdcda] FINISH, SetVdsStatusVDSCommand, log id: 64d54052 >2013-12-12 13:15:06,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [59bdcda] Correlation ID: 59bdcda, Job ID: fa6e0dc1-8616-40c8-8228-28b43edae405, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node2. >2013-12-12 13:15:06,656 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-10) [4152df89] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 Type: VDS >2013-12-12 13:15:06,657 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-10) [4152df89] FINISH, ActivateVdsVDSCommand, return: Host[node2], log id: 6e37ba65 >2013-12-12 13:15:06,675 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [4152df89] Activate finished. Lock released. Monitoring can run now for host node2 from data-center Default >2013-12-12 13:15:06,683 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [4152df89] Correlation ID: 10894266, Call Stack: null, Custom Event ID: -1, Message: Host node2 was autorecovered. >2013-12-12 13:15:06,683 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-10) [4152df89] Lock freed to object EngineLock [exclusiveLocks= key: 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 value: VDS >, sharedLocks= ] >2013-12-12 13:15:49,395 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-73) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:15:49,622 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:15:52,629 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:15:52,632 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:15:52,829 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-83) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.SocketException: Connection reset >2013-12-12 13:15:52,854 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.SocketException: Connection reset >2013-12-12 13:15:55,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:15:55,859 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:15:57,636 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:15:57,638 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:15:58,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:15:58,867 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:16:02,644 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:02,646 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:03,874 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:03,876 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:04,044 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2013-12-12 13:16:04,044 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:16:04,045 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2013-12-12 13:16:04,045 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:16:07,651 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:07,653 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:08,880 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:08,881 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:10,661 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:16:10,662 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:16:13,666 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2013-12-12 13:16:13,667 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Failed to refresh VDS , vds = 273a2ccd-fac3-4ffc-9c0f-e4fbfa054a69 : node2, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2013-12-12 13:16:13,887 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2013-12-12 13:16:13,889 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Failed to refresh VDS , vds = a89c7427-d95e-46b5-bac9-ce80c2dc7f28 : node1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2013-12-12 13:25:54,567 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2013-12-12 13:25:54,570 WARN [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2013-12-12 13:25:54,570 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhsc-doc.conf". >2013-12-12 13:25:54,571 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2013-12-12 13:25:54,571 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2013-12-12 13:25:54,571 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2013-12-12 13:25:54,571 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2013-12-12 13:25:54,571 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-access-plugin.conf". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_AJP_ENABLED" is "true". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_AJP_PORT" is "8702". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_APPS" is "engine.ear redhat-access-plugin-rhsc.ear". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_DATABASE" is "engine". >2013-12-12 13:25:54,585 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_HOST" is "localhost". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_PASSWORD" is "***". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_PORT" is "5432". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_SECURED" is "False". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_SSL" is "false". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_USER" is "engine". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2013-12-12 13:25:54,586 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_FQDN" is "latest.rhsc". >2013-12-12 13:25:54,587 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_GROUP" is "ovirt". >2013-12-12 13:25:54,587 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HEAP_MAX" is "1g". >2013-12-12 13:25:54,601 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HEAP_MIN" is "1g". >2013-12-12 13:25:54,601 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_PORT" is "None". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTP_ENABLED" is "false". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTP_PORT" is "None". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhsc-doc/manual". >2013-12-12 13:25:54,602 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PERM_MAX" is "256m". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PERM_MIN" is "256m". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_ENABLED" is "true". >2013-12-12 13:25:54,611 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_STOP_INTERVAL" is "1". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_STOP_TIME" is "10". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_USER" is "ovirt". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "ENGINE_VERBOSE_GC" is "false". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2013-12-12 13:25:54,612 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool -- 43) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2013-12-12 13:25:56,056 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Start initializing Backend >2013-12-12 13:25:56,847 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Running ovirt-engine 2.1.2-0.27.beta.el6_5 >2013-12-12 13:25:56,848 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Start initializing dictionaries >2013-12-12 13:25:56,857 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Finished initializing dictionaries >2013-12-12 13:25:56,858 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Start initializing AuditLogCleanupManager >2013-12-12 13:25:56,858 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Setting audit cleanup manager to run at: 35 35 3 * * ? >2013-12-12 13:25:56,895 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Finished initializing AuditLogCleanupManager >2013-12-12 13:25:56,897 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Start initializing TagsDirector >2013-12-12 13:25:56,923 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Tag root added to tree >2013-12-12 13:25:56,946 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Finished initializing TagsDirector >2013-12-12 13:25:56,948 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Start initializing IsoDomainListSyncronizer >2013-12-12 13:25:56,962 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Finished initializing IsoDomainListSyncronizer >2013-12-12 13:25:56,983 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 42) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2013-12-12 13:25:57,350 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Completed initializing handlers >2013-12-12 13:25:57,355 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2013-12-12 13:25:57,380 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2013-12-12 13:25:57,381 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2013-12-12 13:25:57,384 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2013-12-12 13:25:57,385 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2013-12-12 13:25:57,385 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Mark incomplete jobs as UNKNOWN >2013-12-12 13:25:57,445 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Start initializing JobRepositoryCleanupManager >2013-12-12 13:25:57,452 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Finished initializing JobRepositoryCleanupManager >2013-12-12 13:25:57,453 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Start initializing AutoRecoveryManager >2013-12-12 13:25:57,458 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Finished initializing AutoRecoveryManager >2013-12-12 13:25:57,459 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Start initializing ExecutionMessageDirector >2013-12-12 13:25:57,470 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Finished initializing ExecutionMessageDirector >2013-12-12 13:25:57,510 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 37) Start initializing UsersDomainsCacheManagerService >2013-12-12 13:25:57,530 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 37) Start initializing DbUserCacheManager >2013-12-12 13:25:57,539 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (ServerService Thread Pool -- 37) Finished initializing DbUserCacheManager >2013-12-12 13:25:57,539 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (ServerService Thread Pool -- 37) Finished initializing UsersDomainsCacheManagerService >2013-12-12 13:25:57,553 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ServerService Thread Pool -- 43) Initialization of AsyncTaskManager completed successfully. >2013-12-12 13:25:57,556 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 43) Start initializing ResourceManager >2013-12-12 13:25:57,584 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2013-12-12 13:25:57,808 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 43) Finished initializing ResourceManager >2013-12-12 13:25:57,820 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 43) Initialization of OvfDataUpdater completed successfully. >2013-12-12 13:25:57,826 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Start scheduling to enable vds load balancer >2013-12-12 13:25:57,827 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Finished scheduling to enable vds load balancer >2013-12-12 13:25:57,850 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-4-thread-1) MacPoolManager(32eeca40): Start initializing >2013-12-12 13:25:57,895 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 43) Init VM custom properties utilities >2013-12-12 13:25:57,931 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-4-thread-1) MacPoolManager(32eeca40): Finished initializing >2013-12-12 13:25:57,960 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 43) Init device custom properties utilities >2013-12-12 13:25:57,967 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Initializing Scheduling manager >2013-12-12 13:25:58,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) External scheduler disabled, discovery skipped >2013-12-12 13:25:58,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Initialized Scheduling manager >2013-12-12 13:25:58,058 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 43) Initializing DWH Heart Beat >2013-12-12 13:26:12,117 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc.json] >2013-12-12 13:26:12,137 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat-access-plugin-rhsc-config.json] >2013-12-12 13:26:54,524 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginAdminUserCommand internal: false. >2013-12-12 13:26:54,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: UNASSIGNED not exist in string table >2013-12-12 13:26:54,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2013-12-12 13:26:54,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2013-12-12 13:26:54,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2013-12-12 13:26:54,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2013-12-12 13:26:54,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: DWH_STOPPED not exist in string table >2013-12-12 13:26:54,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: DWH_STARTED not exist in string table >2013-12-12 13:26:54,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: DWH_ERROR not exist in string table >2013-12-12 13:26:54,645 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: GLUSTER_VOLUME_REBALANCE_START_DETECTED_FROM_CLI not have severity. Assumed Normal >2013-12-12 13:26:54,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: START_REMOVING_GLUSTER_VOLUME_BRICKS_DETECTED_FROM_CLI not have severity. Assumed Normal >2013-12-12 13:26:54,647 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2013-12-12 13:26:54,649 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2013-12-12 13:26:54,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:26:57,715 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:26:57,741 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:26:57,991 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-43) No up server in cluster >2013-12-12 13:26:57,992 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-43) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:27:57,996 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-87) No up server in cluster >2013-12-12 13:27:57,997 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-87) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:28:57,999 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-30) No up server in cluster >2013-12-12 13:28:57,999 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-30) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:29:58,003 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-73) No up server in cluster >2013-12-12 13:29:58,003 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-73) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:30:58,005 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-19) No up server in cluster >2013-12-12 13:30:58,005 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-19) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:31:58,008 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-62) No up server in cluster >2013-12-12 13:31:58,008 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-62) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:32:54,643 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:32:54,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:32:54,704 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [4c69bdbb] Running command: LogoutUserCommand internal: false. >2013-12-12 13:32:54,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [4c69bdbb] Correlation ID: 4c69bdbb, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:32:56,767 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:32:56,781 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:32:57,619 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-10) [23ee8e0c] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:32:57,714 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [23ee8e0c] Correlation ID: 23ee8e0c, Call Stack: null, Custom Event ID: -1, Message: Host cluster myCluster was added by admin@internal >2013-12-12 13:32:57,727 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [7030ec49] Running command: LogoutUserCommand internal: false. >2013-12-12 13:32:57,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [7030ec49] Correlation ID: 7030ec49, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:32:58,012 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-5) No up server in cluster >2013-12-12 13:32:58,012 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-5) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:32:58,013 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-5) No up server in cluster >2013-12-12 13:32:58,014 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-5) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:32:59,017 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:32:59,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:32:59,077 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [6dd15e9c] Running command: LogoutUserCommand internal: false. >2013-12-12 13:32:59,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [6dd15e9c] Correlation ID: 6dd15e9c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:00,441 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:00,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:02,015 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-2) [72d47b1e] Running command: AddVdsCommand internal: false. Entities affected : ID: 40302705-a79d-416e-b099-68670bf5637f Type: VdsGroups >2013-12-12 13:33:02,142 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [7688033a] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 13:33:02,173 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [7688033a] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 13:33:02,183 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [7688033a] Cannot get vdsManager for vdsid=a77a7c9d-6a77-4f57-8db1-8162e4d58fff >2013-12-12 13:33:02,214 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] START, RemoveVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a8a01be >2013-12-12 13:33:02,214 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [7688033a] Cannot get vdsManager for vdsid=a77a7c9d-6a77-4f57-8db1-8162e4d58fff >2013-12-12 13:33:02,215 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] FINISH, RemoveVdsVDSCommand, log id: 4a8a01be >2013-12-12 13:33:02,216 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [7688033a] Cannot get vdsManager for vdsid=a77a7c9d-6a77-4f57-8db1-8162e4d58fff >2013-12-12 13:33:02,221 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] START, AddVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 79a87d9c >2013-12-12 13:33:02,221 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] AddVds - entered , starting logic to add VDS a77a7c9d-6a77-4f57-8db1-8162e4d58fff >2013-12-12 13:33:02,222 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] AddVds - VDS a77a7c9d-6a77-4f57-8db1-8162e4d58fff was added, will try to add it to the resource manager >2013-12-12 13:33:02,226 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [7688033a] Entered VdsManager constructor >2013-12-12 13:33:02,262 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [7688033a] Initialize vdsBroker (latest-a,54,321) >2013-12-12 13:33:02,523 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [7688033a] VDS a77a7c9d-6a77-4f57-8db1-8162e4d58fff was added to the Resource Manager >2013-12-12 13:33:02,525 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [7688033a] FINISH, AddVdsVDSCommand, log id: 79a87d9c >2013-12-12 13:33:02,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [7688033a] Correlation ID: 72d47b1e, Job ID: a6910fca-ba0e-47ad-91f8-0788e0789917, Call Stack: null, Custom Event ID: -1, Message: Host node1 was added by admin@internal. >2013-12-12 13:33:02,716 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [2e935650] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:02,785 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-8) [6a489715] Lock Acquired to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 13:33:02,798 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [2e935650] Correlation ID: 2e935650, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:02,815 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-8) [6a489715] Running command: InstallVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 13:33:02,815 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-8) [6a489715] Before Installation host a77a7c9d-6a77-4f57-8db1-8162e4d58fff, node1 >2013-12-12 13:33:02,873 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-8) [6a489715] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Installing, nonOperationalReason=NONE), log id: 762cd338 >2013-12-12 13:33:02,906 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-8) [6a489715] FINISH, SetVdsStatusVDSCommand, log id: 762cd338 >2013-12-12 13:33:03,010 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-8) [6a489715] Installation latest-a: Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 13:33:03,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-8) [6a489715] Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Connected to host latest-a with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 13:33:03,051 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-8) [6a489715] Installation of latest-a. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 13:33:03,063 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-8) SSH execute root@latest-a 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 13:33:03,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Initializing >2013-12-12 13:33:03,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Initializing. >2013-12-12 13:33:03,916 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment setup >2013-12-12 13:33:03,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment setup. >2013-12-12 13:33:03,934 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment packages setup >2013-12-12 13:33:03,956 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment packages setup. >2013-12-12 13:33:04,619 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Programs detection >2013-12-12 13:33:04,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Programs detection. >2013-12-12 13:33:04,632 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Environment customization >2013-12-12 13:33:04,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Environment customization. >2013-12-12 13:33:04,668 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Logs at host located at: '/tmp/ovirt-host-deploy-20131212133304.log' >2013-12-12 13:33:04,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Logs at host located at: '/tmp/ovirt-host-deploy-20131212133304.log'. >2013-12-12 13:33:04,738 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-a reports unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 >2013-12-12 13:33:04,765 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 132493DB-4ACB-4347-178C-B16C2E38EFE9 to Host latest-a >2013-12-12 13:33:04,860 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Setup validation >2013-12-12 13:33:04,873 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Setup validation. >2013-12-12 13:33:05,010 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a' >2013-12-12 13:33:05,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Host node1 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-a'. >2013-12-12 13:33:05,027 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction setup >2013-12-12 13:33:05,040 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Transaction setup. >2013-12-12 13:33:05,058 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 13:33:05,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Misc configuration. >2013-12-12 13:33:05,073 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Package installation >2013-12-12 13:33:05,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Package installation. >2013-12-12 13:33:05,212 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:05,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:05,247 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [4f297b1f] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:05,256 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [4f297b1f] Correlation ID: 4f297b1f, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:05,525 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Initializing Host: node1 >2013-12-12 13:33:06,725 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:06,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:06,754 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [39f6357b] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:06,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [39f6357b] Correlation ID: 39f6357b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:08,358 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:08,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:08,399 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [52b700bb] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:08,406 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [52b700bb] Correlation ID: 52b700bb, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:08,775 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Misc configuration >2013-12-12 13:33:08,807 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Misc configuration. >2013-12-12 13:33:08,823 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Setting time >2013-12-12 13:33:08,840 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Setting time. >2013-12-12 13:33:09,960 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:09,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:10,003 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [5eeb7d85] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:10,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [5eeb7d85] Correlation ID: 5eeb7d85, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:11,490 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:11,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:11,525 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [18598271] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:11,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [18598271] Correlation ID: 18598271, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:11,625 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Enrolling certificate >2013-12-12 13:33:11,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Enrolling certificate. >2013-12-12 13:33:12,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Transaction commit >2013-12-12 13:33:12,990 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Transaction commit. >2013-12-12 13:33:13,000 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Closing up >2013-12-12 13:33:13,094 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:13,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Closing up. >2013-12-12 13:33:13,094 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting gluster >2013-12-12 13:33:13,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Starting gluster. >2013-12-12 13:33:13,123 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:13,134 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [6f0ee819] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:13,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [6f0ee819] Correlation ID: 6f0ee819, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:14,672 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:14,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:14,708 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [11b8b940] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:14,723 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [11b8b940] Correlation ID: 11b8b940, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:16,406 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:16,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:16,428 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [770a9999] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:16,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [770a9999] Correlation ID: 770a9999, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:17,860 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:17,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:17,959 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [1f3f6b1b] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:17,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1f3f6b1b] Correlation ID: 1f3f6b1b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:19,078 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Starting vdsm >2013-12-12 13:33:19,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Starting vdsm. >2013-12-12 13:33:19,448 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:19,467 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:19,474 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [5a1f1972] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:19,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [5a1f1972] Correlation ID: 5a1f1972, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:20,998 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:21,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:21,025 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [bc175d2] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:21,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [bc175d2] Correlation ID: bc175d2, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:21,737 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Pre-termination >2013-12-12 13:33:21,750 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Pre-termination. >2013-12-12 13:33:22,520 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:22,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:22,553 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [5ae27d3b] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:22,563 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212133322-latest-a-6a489715.log' >2013-12-12 13:33:22,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212133322-latest-a-6a489715.log'. >2013-12-12 13:33:22,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [5ae27d3b] Correlation ID: 5ae27d3b, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:23,167 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-a: Stage: Termination >2013-12-12 13:33:23,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6a489715, Call Stack: null, Custom Event ID: -1, Message: Installing Host node1. Stage: Termination. >2013-12-12 13:33:23,592 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-8) [6a489715] Engine managed to communicate with VDSM agent on host node1 >2013-12-12 13:33:24,051 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:24,160 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:24,174 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [5fc44c06] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:24,176 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-8) [6a489715] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Initializing, nonOperationalReason=NONE), log id: 31ef281c >2013-12-12 13:33:24,184 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-8) [6a489715] FINISH, SetVdsStatusVDSCommand, log id: 31ef281c >2013-12-12 13:33:24,184 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-8) [6a489715] After Installation host node1, VDS >2013-12-12 13:33:24,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [5fc44c06] Correlation ID: 5fc44c06, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:24,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-8) [6a489715] Correlation ID: 6a489715, Job ID: a6910fca-ba0e-47ad-91f8-0788e0789917, Call Stack: null, Custom Event ID: -1, Message: Host node1 installed >2013-12-12 13:33:24,233 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-8) [6a489715] Lock freed to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 13:33:25,623 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:25,642 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:25,650 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [13fe2703] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:25,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [13fe2703] Correlation ID: 13fe2703, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:26,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 44f612ab >2013-12-12 13:33:26,684 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 44f612ab >2013-12-12 13:33:26,752 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 44f9d389 >2013-12-12 13:33:27,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 44f9d389 >2013-12-12 13:33:27,278 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:27,285 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 554fa13c >2013-12-12 13:33:27,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:27,311 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 554fa13c >2013-12-12 13:33:27,325 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [432c7b5e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:27,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [432c7b5e] Correlation ID: 432c7b5e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:27,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 673e6b2a >2013-12-12 13:33:27,703 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 673e6b2a >2013-12-12 13:33:27,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host node1. >2013-12-12 13:33:27,766 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-30) [5a72bd77] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:33:27,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-30) [5a72bd77] START, GetGlusterHostUUIDVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6a620a48 >2013-12-12 13:33:27,957 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-30) [5a72bd77] FINISH, GetGlusterHostUUIDVDSCommand, return: bac2fdd4-c09f-466b-b977-bf240c911146, log id: 6a620a48 >2013-12-12 13:33:28,173 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 13:33:28,465 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1e113476 >2013-12-12 13:33:28,812 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 1e113476 >2013-12-12 13:33:28,815 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6fc219e5 >2013-12-12 13:33:29,061 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6fc219e5 >2013-12-12 13:33:29,114 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:29,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:29,233 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [42567e88] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:29,241 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [42567e88] Correlation ID: 42567e88, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:30,038 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:30,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:30,066 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [55953184] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:30,075 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [55953184] Correlation ID: 55953184, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:30,623 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:30,650 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:30,657 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [53390121] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:30,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [53390121] Correlation ID: 53390121, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:31,252 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:31,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:31,298 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [324abf4d] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:31,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [324abf4d] Correlation ID: 324abf4d, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:31,994 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:32,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:32,550 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-9) [5969d5d] Running command: AddVdsCommand internal: false. Entities affected : ID: 40302705-a79d-416e-b099-68670bf5637f Type: VdsGroups >2013-12-12 13:33:32,568 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [38a17851] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 13:33:32,583 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [38a17851] Lock freed to object EngineLock [exclusiveLocks= key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: REGISTER_VDS >, sharedLocks= ] >2013-12-12 13:33:32,584 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [38a17851] Cannot get vdsManager for vdsid=60cc0e1d-bcb1-429b-9a15-aab41a70dc20 >2013-12-12 13:33:32,586 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] START, RemoveVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: f7e1581 >2013-12-12 13:33:32,586 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [38a17851] Cannot get vdsManager for vdsid=60cc0e1d-bcb1-429b-9a15-aab41a70dc20 >2013-12-12 13:33:32,586 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] FINISH, RemoveVdsVDSCommand, log id: f7e1581 >2013-12-12 13:33:32,586 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [38a17851] Cannot get vdsManager for vdsid=60cc0e1d-bcb1-429b-9a15-aab41a70dc20 >2013-12-12 13:33:32,587 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] START, AddVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 69019eab >2013-12-12 13:33:32,587 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] AddVds - entered , starting logic to add VDS 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 >2013-12-12 13:33:32,593 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] AddVds - VDS 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 was added, will try to add it to the resource manager >2013-12-12 13:33:32,594 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [38a17851] Entered VdsManager constructor >2013-12-12 13:33:32,621 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [38a17851] Initialize vdsBroker (latest-b,54,321) >2013-12-12 13:33:32,623 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [38a17851] VDS 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 was added to the Resource Manager >2013-12-12 13:33:32,623 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [38a17851] FINISH, AddVdsVDSCommand, log id: 69019eab >2013-12-12 13:33:32,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [38a17851] Correlation ID: 5969d5d, Job ID: 4c86611c-8991-4a8f-b8f7-eb3c89b918a7, Call Stack: null, Custom Event ID: -1, Message: Host node2 was added by admin@internal. >2013-12-12 13:33:32,667 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-43) [28618466] Lock Acquired to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 13:33:32,673 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [f60b6a5] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:32,683 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [f60b6a5] Correlation ID: f60b6a5, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:32,691 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-43) [28618466] Running command: InstallVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 13:33:32,691 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-43) [28618466] Before Installation host 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, node2 >2013-12-12 13:33:32,693 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-43) [28618466] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Installing, nonOperationalReason=NONE), log id: 22e7fde5 >2013-12-12 13:33:32,708 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-43) [28618466] FINISH, SetVdsStatusVDSCommand, log id: 22e7fde5 >2013-12-12 13:33:32,782 INFO [org.ovirt.engine.core.bll.InstallerMessages] (pool-4-thread-43) [28618466] Installation latest-b: Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2 >2013-12-12 13:33:32,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-43) [28618466] Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Connected to host latest-b with SSH key fingerprint: 37:6b:cd:9f:10:78:a0:9f:84:fc:36:11:2b:d0:03:a2. >2013-12-12 13:33:32,807 INFO [org.ovirt.engine.core.bll.VdsDeploy] (pool-4-thread-43) [28618466] Installation of latest-b. Executing command via SSH umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2013-12-12 13:33:32,816 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (pool-4-thread-43) SSH execute root@latest-b 'umask 0077; MYTMP="$(mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2013-12-12 13:33:33,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Initializing >2013-12-12 13:33:33,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Initializing. >2013-12-12 13:33:33,541 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment setup >2013-12-12 13:33:33,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment setup. >2013-12-12 13:33:33,559 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment packages setup >2013-12-12 13:33:33,567 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment packages setup. >2013-12-12 13:33:34,075 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5957f41a >2013-12-12 13:33:34,189 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:34,192 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Programs detection >2013-12-12 13:33:34,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Programs detection. >2013-12-12 13:33:34,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Environment customization >2013-12-12 13:33:34,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:34,305 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [3c18257c] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:34,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Environment customization. >2013-12-12 13:33:34,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [3c18257c] Correlation ID: 3c18257c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:34,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Logs at host located at: '/tmp/ovirt-host-deploy-20131212133333.log' >2013-12-12 13:33:34,393 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Logs at host located at: '/tmp/ovirt-host-deploy-20131212133333.log'. >2013-12-12 13:33:34,422 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 5957f41a >2013-12-12 13:33:34,423 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4c6d4b06 >2013-12-12 13:33:34,460 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host latest-b reports unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 >2013-12-12 13:33:34,462 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 7B904C94-94EE-7B49-40B3-BD3C28FBB749 to Host latest-b >2013-12-12 13:33:34,556 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Setup validation >2013-12-12 13:33:34,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Setup validation. >2013-12-12 13:33:34,615 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 4c6d4b06 >2013-12-12 13:33:34,702 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b' >2013-12-12 13:33:34,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Host node2 installation in progress . Cannot validate host name settings, reason: cannot resolve own name 'rhs-21u2-20131208-b'. >2013-12-12 13:33:34,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction setup >2013-12-12 13:33:34,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Transaction setup. >2013-12-12 13:33:34,750 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 13:33:34,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Misc configuration. >2013-12-12 13:33:34,761 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Package installation >2013-12-12 13:33:34,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Package installation. >2013-12-12 13:33:35,625 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Initializing Host: node2 >2013-12-12 13:33:36,199 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:36,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:36,230 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [104977d1] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:36,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [104977d1] Correlation ID: 104977d1, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:36,502 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Misc configuration >2013-12-12 13:33:36,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Misc configuration. >2013-12-12 13:33:36,580 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Setting time >2013-12-12 13:33:36,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Setting time. >2013-12-12 13:33:37,685 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:37,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:37,722 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [37388e25] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:37,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [37388e25] Correlation ID: 37388e25, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:38,333 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Enrolling certificate >2013-12-12 13:33:38,345 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Enrolling certificate. >2013-12-12 13:33:39,189 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:39,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:39,333 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [2603b291] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:39,344 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [2603b291] Correlation ID: 2603b291, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:39,632 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7f91a9ac >2013-12-12 13:33:39,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Transaction commit >2013-12-12 13:33:39,724 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Transaction commit. >2013-12-12 13:33:39,724 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Closing up >2013-12-12 13:33:39,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Closing up. >2013-12-12 13:33:39,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting gluster >2013-12-12 13:33:39,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Starting gluster. >2013-12-12 13:33:39,979 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 7f91a9ac >2013-12-12 13:33:39,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7583e652 >2013-12-12 13:33:40,163 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 7583e652 >2013-12-12 13:33:40,825 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:40,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:40,866 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [735c549a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:40,878 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [735c549a] Correlation ID: 735c549a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:43,343 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:43,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:43,427 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [2318a69] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:43,501 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2318a69] Correlation ID: 2318a69, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:45,042 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:45,071 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:45,080 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [b69ea46] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:45,089 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [b69ea46] Correlation ID: b69ea46, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:45,177 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 741fcb8d >2013-12-12 13:33:45,515 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 741fcb8d >2013-12-12 13:33:45,517 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6af37d57 >2013-12-12 13:33:45,598 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Starting vdsm >2013-12-12 13:33:45,613 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Starting vdsm. >2013-12-12 13:33:45,695 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 6af37d57 >2013-12-12 13:33:46,559 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:46,588 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:46,598 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [59664564] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:46,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [59664564] Correlation ID: 59664564, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:48,158 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Pre-termination >2013-12-12 13:33:48,171 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Pre-termination. >2013-12-12 13:33:48,232 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:48,255 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:48,261 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [1688cc4e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:48,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [1688cc4e] Correlation ID: 1688cc4e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:48,991 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212133348-latest-b-28618466.log' >2013-12-12 13:33:49,005 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20131212133348-latest-b-28618466.log'. >2013-12-12 13:33:49,536 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation latest-b: Stage: Termination >2013-12-12 13:33:49,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 28618466, Call Stack: null, Custom Event ID: -1, Message: Installing Host node2. Stage: Termination. >2013-12-12 13:33:49,816 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:49,824 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (pool-4-thread-43) [28618466] Engine managed to communicate with VDSM agent on host node2 >2013-12-12 13:33:49,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:49,846 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [68cbc2d9] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:49,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [68cbc2d9] Correlation ID: 68cbc2d9, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:50,240 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-43) [28618466] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Initializing, nonOperationalReason=NONE), log id: 786436f8 >2013-12-12 13:33:50,255 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-43) [28618466] FINISH, SetVdsStatusVDSCommand, log id: 786436f8 >2013-12-12 13:33:50,255 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-43) [28618466] After Installation host node2, VDS >2013-12-12 13:33:50,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-43) [28618466] Correlation ID: 28618466, Job ID: 4c86611c-8991-4a8f-b8f7-eb3c89b918a7, Call Stack: null, Custom Event ID: -1, Message: Host node2 installed >2013-12-12 13:33:50,288 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-43) [28618466] Lock freed to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 13:33:50,707 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-61) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3b4d7a34 >2013-12-12 13:33:50,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: 533f6c01 >2013-12-12 13:33:50,745 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GetHardwareInfoVDSCommand, log id: 533f6c01 >2013-12-12 13:33:50,774 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7d3722c4 >2013-12-12 13:33:51,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 3b4d7a34 >2013-12-12 13:33:51,052 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 78612e5 >2013-12-12 13:33:51,198 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED], log id: 7d3722c4 >2013-12-12 13:33:51,225 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 78612e5 >2013-12-12 13:33:51,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: a63d737 >2013-12-12 13:33:51,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GetHardwareInfoVDSCommand, log id: a63d737 >2013-12-12 13:33:51,274 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 56e83e5e >2013-12-12 13:33:51,305 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:51,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:51,338 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [20dde58a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:51,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [20dde58a] Correlation ID: 20dde58a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:51,621 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED], log id: 56e83e5e >2013-12-12 13:33:51,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host node2. >2013-12-12 13:33:51,688 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-60) [73566704] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-3ce782d831b3 Type: StoragePool >2013-12-12 13:33:51,690 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] START, GetGlusterHostUUIDVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5abadf30 >2013-12-12 13:33:51,863 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] FINISH, GetGlusterHostUUIDVDSCommand, return: 553780c3-c742-4515-9861-4b90c5044dc4, log id: 5abadf30 >2013-12-12 13:33:51,883 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2d06d036 >2013-12-12 13:33:52,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED], log id: 2d06d036 >2013-12-12 13:33:52,224 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] START, AddGlusterServerVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1ceb2fa5 >2013-12-12 13:33:52,780 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:52,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:52,831 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [71ce199a] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:52,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [71ce199a] Correlation ID: 71ce199a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:53,729 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (DefaultQuartzScheduler_Worker-60) [73566704] FINISH, AddGlusterServerVDSCommand, log id: 1ceb2fa5 >2013-12-12 13:33:53,827 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 13:33:54,380 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:54,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:54,552 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [87ec8d8] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:54,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [87ec8d8] Correlation ID: 87ec8d8, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:56,237 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 47ec90fb >2013-12-12 13:33:56,582 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 47ec90fb >2013-12-12 13:33:56,585 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4c8423a1 >2013-12-12 13:33:56,761 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 4c8423a1 >2013-12-12 13:33:57,450 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:57,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:57,477 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [4be28de] Running command: LogoutUserCommand internal: false. >2013-12-12 13:33:57,488 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [4be28de] Correlation ID: 4be28de, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:33:58,016 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-72) No up server in cluster >2013-12-12 13:33:58,016 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-72) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:33:58,022 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6bc7f017 >2013-12-12 13:33:58,203 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterTasksListVDSCommand, return: [], log id: 6bc7f017 >2013-12-12 13:33:58,673 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2013-12-12 13:33:58,796 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:33:59,505 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [7eca50c9] Running command: CreateGlusterVolumeCommand internal: false. Entities affected : ID: 40302705-a79d-416e-b099-68670bf5637f Type: VdsGroups >2013-12-12 13:33:59,507 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [7eca50c9] START, CreateGlusterVolumeVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5459cd4c >2013-12-12 13:34:01,767 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-78) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:02,901 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-78) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:03,432 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-7) [7eca50c9] FINISH, CreateGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@74d21398, log id: 5459cd4c >2013-12-12 13:34:03,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [7eca50c9] Correlation ID: 7eca50c9, Job ID: 7da72f7c-52c1-4bd7-9c9f-a2ed47e829ea, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest created. >2013-12-12 13:34:03,563 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-7) [7eca50c9] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:03,566 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 462edf25 >2013-12-12 13:34:03,593 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [774a91c5] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:03,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [774a91c5] Correlation ID: 774a91c5, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:03,903 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 462edf25 >2013-12-12 13:34:03,905 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4a945749 >2013-12-12 13:34:04,095 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@72094e0c}, log id: 4a945749 >2013-12-12 13:34:04,654 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:04,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:04,679 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [46e2bc6e] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:04,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [46e2bc6e] Correlation ID: 46e2bc6e, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:05,144 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:05,171 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:05,838 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [4c7513db] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:34:05,841 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [4c7513db] START, StartGlusterVolumeVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 24cd12a >2013-12-12 13:34:09,132 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-88) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:10,367 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [4c7513db] FINISH, StartGlusterVolumeVDSCommand, log id: 24cd12a >2013-12-12 13:34:10,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [4c7513db] Correlation ID: 4c7513db, Job ID: bd942fcf-0854-447d-bc64-0c64face6425, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest started. >2013-12-12 13:34:10,458 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [4c7513db] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:10,461 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 54b093ea >2013-12-12 13:34:10,473 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [62362a12] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:10,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [62362a12] Correlation ID: 62362a12, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:10,798 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 54b093ea >2013-12-12 13:34:10,800 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1676c0f2 >2013-12-12 13:34:10,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@62d6c574}, log id: 1676c0f2 >2013-12-12 13:34:16,000 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 33962581 >2013-12-12 13:34:16,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 33962581 >2013-12-12 13:34:16,343 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 45497241 >2013-12-12 13:34:16,523 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@96d360ca}, log id: 45497241 >2013-12-12 13:34:18,544 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:18,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:19,300 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [4e9bf20f] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:34:19,302 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [4e9bf20f] START, AddBricksToGlusterVolumeVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6d0150bd >2013-12-12 13:34:21,541 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-2) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:23,069 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-2) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:24,369 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-2) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:25,907 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-4) [4e9bf20f] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 6d0150bd >2013-12-12 13:34:25,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4e9bf20f] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:34:25,915 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4e9bf20f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212133418659927299489] on host [latest-b] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:34:25,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4e9bf20f] Correlation ID: 4e9bf20f, Job ID: 6a649f8d-2948-4e76-9d17-9d57635f36b0, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:34:25,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 44f4806b >2013-12-12 13:34:25,958 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-4) [4e9bf20f] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:25,973 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [747f0128] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:25,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [747f0128] Correlation ID: 747f0128, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:26,315 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 44f4806b >2013-12-12 13:34:26,317 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4b852ef4 >2013-12-12 13:34:26,497 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@75920987}, log id: 4b852ef4 >2013-12-12 13:34:26,992 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:27,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:27,079 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-9) [2160d901] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:34:27,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-9) [2160d901] START, AddBricksToGlusterVolumeVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4bfb30c7 >2013-12-12 13:34:31,515 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:32,119 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:33,410 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-15) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:33,586 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-9) [2160d901] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 4bfb30c7 >2013-12-12 13:34:33,587 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2160d901] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:34:33,588 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2160d901] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212133427138341374234] on host [latest-a] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:34:33,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2160d901] Correlation ID: 2160d901, Job ID: 9ac8b47a-2fb3-466d-b0c2-eb77f82a98e8, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:34:33,646 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-9) [2160d901] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:33,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 54f0480b >2013-12-12 13:34:33,655 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [1d54c19c] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:33,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [1d54c19c] Correlation ID: 1d54c19c, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:33,996 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 54f0480b >2013-12-12 13:34:33,998 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 29d8608d >2013-12-12 13:34:34,181 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db8cb976}, log id: 29d8608d >2013-12-12 13:34:34,221 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:34,247 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:34,421 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-12) [3d7b19fe] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:34:34,422 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-12) [3d7b19fe] START, AddBricksToGlusterVolumeVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2a46d71c >2013-12-12 13:34:39,212 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:39,612 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:41,372 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:42,591 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-12) [3d7b19fe] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 2a46d71c >2013-12-12 13:34:42,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [3d7b19fe] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:34:42,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [3d7b19fe] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/20131212133434230219930446] on host [latest-b] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:34:42,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [3d7b19fe] Correlation ID: 3d7b19fe, Job ID: 39570284-faae-463f-b2a2-2426deeb9538, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:34:42,630 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-25) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:42,638 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-12) [3d7b19fe] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:42,639 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 66722a30 >2013-12-12 13:34:42,644 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [4f80d960] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:42,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [4f80d960] Correlation ID: 4f80d960, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:43,099 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 66722a30 >2013-12-12 13:34:43,105 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7ab0cad3 >2013-12-12 13:34:43,281 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f44e5b30}, log id: 7ab0cad3 >2013-12-12 13:34:43,508 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2013-12-12 13:34:43,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:34:43,696 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [492b2e51] Running command: AddBricksToGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:34:43,697 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [492b2e51] START, AddBricksToGlusterVolumeVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 8f84595 >2013-12-12 13:34:48,301 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-35) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:48,653 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-35) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:50,425 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-35) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:50,978 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddBricksToGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-6) [492b2e51] FINISH, AddBricksToGlusterVolumeVDSCommand, log id: 8f84595 >2013-12-12 13:34:50,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [492b2e51] No severity for GLUSTER_VOLUME_BRICK_ADDED audit log type, assuming Normal severity >2013-12-12 13:34:50,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [492b2e51] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Brick [/bricks/201312121334434851151947411] on host [latest-a] added to volume [StartMigrationDuringRebalanceTest] >2013-12-12 13:34:51,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [492b2e51] Correlation ID: 492b2e51, Job ID: 185fdb1d-86ed-4c1f-aa30-a5029246fdbe, Call Stack: null, Custom Event ID: -1, Message: 1 brick(s) added to volume StartMigrationDuringRebalanceTest. >2013-12-12 13:34:51,013 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5fb64c50 >2013-12-12 13:34:51,014 INFO [org.ovirt.engine.core.bll.gluster.AddBricksToGlusterVolumeCommand] (ajp-/127.0.0.1:8702-6) [492b2e51] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:34:51,020 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [49e0fba0] Running command: LogoutUserCommand internal: false. >2013-12-12 13:34:51,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [49e0fba0] Correlation ID: 49e0fba0, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 13:34:51,486 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5fb64c50 >2013-12-12 13:34:51,489 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7d436023 >2013-12-12 13:34:51,680 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5bed9901}, log id: 7d436023 >2013-12-12 13:34:56,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 69a103e1 >2013-12-12 13:34:57,035 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 69a103e1 >2013-12-12 13:34:57,037 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21d50751 >2013-12-12 13:34:57,223 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@729a3d5a}, log id: 21d50751 >2013-12-12 13:34:58,209 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-52) No up server in cluster >2013-12-12 13:34:58,209 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-52) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:34:58,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5aabcec4 >2013-12-12 13:34:58,391 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5aabcec4 >2013-12-12 13:35:02,240 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3830273c >2013-12-12 13:35:02,579 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3830273c >2013-12-12 13:35:02,581 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: ccb50bb >2013-12-12 13:35:02,762 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@64ffa8fc}, log id: ccb50bb >2013-12-12 13:35:07,780 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3ddf7c10 >2013-12-12 13:35:08,115 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3ddf7c10 >2013-12-12 13:35:08,117 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6abfcd33 >2013-12-12 13:35:08,308 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c6447940}, log id: 6abfcd33 >2013-12-12 13:35:08,454 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-43) [35f9eb6c] Running command: StopGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:35:08,456 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-43) [35f9eb6c] START, StopGlusterVolumeVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4874609 >2013-12-12 13:35:13,326 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:14,680 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-73) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:15,611 INFO [org.ovirt.engine.core.vdsbroker.gluster.StopGlusterVolumeVDSCommand] (pool-4-thread-43) [35f9eb6c] FINISH, StopGlusterVolumeVDSCommand, log id: 4874609 >2013-12-12 13:35:15,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-43) [35f9eb6c] Correlation ID: 35f9eb6c, Job ID: 7131f5eb-2a84-4a0f-a7f0-d57f9becd4e4, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest stopped. >2013-12-12 13:35:15,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1e036da1 >2013-12-12 13:35:15,730 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (pool-4-thread-43) [35f9eb6c] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:16,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1e036da1 >2013-12-12 13:35:16,084 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 331f4897 >2013-12-12 13:35:16,267 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@313d90ef}, log id: 331f4897 >2013-12-12 13:35:21,285 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 53d0f5a7 >2013-12-12 13:35:21,622 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 53d0f5a7 >2013-12-12 13:35:21,626 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 461cd0d >2013-12-12 13:35:21,805 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e1d8befd}, log id: 461cd0d >2013-12-12 13:35:23,987 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-43) [e418061] Running command: StartGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 13:35:23,988 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-43) [e418061] START, StartGlusterVolumeVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 72afaedf >2013-12-12 13:35:26,820 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-87) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:28,071 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-87) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:29,776 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartGlusterVolumeVDSCommand] (pool-4-thread-43) [e418061] FINISH, StartGlusterVolumeVDSCommand, log id: 72afaedf >2013-12-12 13:35:29,805 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-87) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:29,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-4-thread-43) [e418061] Correlation ID: e418061, Job ID: aea5263d-dc19-4900-9f2c-18113b24f21c, Call Stack: null, Custom Event ID: -1, Message: Gluster Volume StartMigrationDuringRebalanceTest started. >2013-12-12 13:35:29,925 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 38d1fc8 >2013-12-12 13:35:29,926 INFO [org.ovirt.engine.core.bll.gluster.StartGlusterVolumeCommand] (pool-4-thread-43) [e418061] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:30,362 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 38d1fc8 >2013-12-12 13:35:30,365 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6b364567 >2013-12-12 13:35:30,551 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9e6326e4}, log id: 6b364567 >2013-12-12 13:35:35,569 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 50e8e859 >2013-12-12 13:35:35,904 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 50e8e859 >2013-12-12 13:35:35,906 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7958a19b >2013-12-12 13:35:36,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@faa6c422}, log id: 7958a19b >2013-12-12 13:35:41,104 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 356185c7 >2013-12-12 13:35:41,445 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 356185c7 >2013-12-12 13:35:41,447 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 43e857c4 >2013-12-12 13:35:41,630 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@de7cc259}, log id: 43e857c4 >2013-12-12 13:35:46,652 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 28e2f152 >2013-12-12 13:35:46,992 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 28e2f152 >2013-12-12 13:35:46,994 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 34c73e29 >2013-12-12 13:35:47,171 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7e9da739}, log id: 34c73e29 >2013-12-12 13:35:52,197 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 319c7188 >2013-12-12 13:35:52,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 319c7188 >2013-12-12 13:35:52,535 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: baa74b7 >2013-12-12 13:35:52,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@68ce645f}, log id: baa74b7 >2013-12-12 13:35:57,735 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 15e07c6d >2013-12-12 13:35:57,869 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-33) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:58,078 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 15e07c6d >2013-12-12 13:35:58,079 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-33) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 13:35:58,080 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 626dbcca >2013-12-12 13:35:58,257 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4e748d64}, log id: 626dbcca >2013-12-12 13:35:58,269 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 516d8793 >2013-12-12 13:35:58,395 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-38) No up server in cluster >2013-12-12 13:35:58,395 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-38) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:35:58,397 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5966b715 >2013-12-12 13:35:58,530 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@180ed3c9, log id: 516d8793 >2013-12-12 13:35:58,627 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5966b715 >2013-12-12 13:36:03,279 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7387a412 >2013-12-12 13:36:03,612 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7387a412 >2013-12-12 13:36:03,614 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 43e0c59f >2013-12-12 13:36:03,793 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1f3d2709}, log id: 43e0c59f >2013-12-12 13:36:08,815 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4fd5e7b0 >2013-12-12 13:36:09,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4fd5e7b0 >2013-12-12 13:36:09,168 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 523076c6 >2013-12-12 13:36:09,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b7dab80e}, log id: 523076c6 >2013-12-12 13:36:14,363 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1e526a2a >2013-12-12 13:36:14,701 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1e526a2a >2013-12-12 13:36:14,703 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: db20ff2 >2013-12-12 13:36:14,882 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6e3a89ce}, log id: db20ff2 >2013-12-12 13:36:19,894 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7c025f0b >2013-12-12 13:36:20,231 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7c025f0b >2013-12-12 13:36:20,233 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4a27dde0 >2013-12-12 13:36:20,411 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bb533798}, log id: 4a27dde0 >2013-12-12 13:36:25,425 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a256675 >2013-12-12 13:36:25,765 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4a256675 >2013-12-12 13:36:25,767 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5a710110 >2013-12-12 13:36:25,947 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@468ea72b}, log id: 5a710110 >2013-12-12 13:36:30,961 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 603e838d >2013-12-12 13:36:31,298 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 603e838d >2013-12-12 13:36:31,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3340bf8f >2013-12-12 13:36:31,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cb1065c8}, log id: 3340bf8f >2013-12-12 13:36:36,491 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5745b5f7 >2013-12-12 13:36:36,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5745b5f7 >2013-12-12 13:36:36,835 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: b9e3ab7 >2013-12-12 13:36:37,023 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@61bf77f7}, log id: b9e3ab7 >2013-12-12 13:36:42,041 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 442a145f >2013-12-12 13:36:42,372 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 442a145f >2013-12-12 13:36:42,374 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1e8c13b8 >2013-12-12 13:36:42,552 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e9061760}, log id: 1e8c13b8 >2013-12-12 13:36:47,567 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 9eb6cf0 >2013-12-12 13:36:47,904 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 9eb6cf0 >2013-12-12 13:36:47,906 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: c891970 >2013-12-12 13:36:48,085 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6acafbb2}, log id: c891970 >2013-12-12 13:36:53,099 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4b19bfda >2013-12-12 13:36:53,443 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4b19bfda >2013-12-12 13:36:53,445 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 162c2db0 >2013-12-12 13:36:53,623 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c30e88a4}, log id: 162c2db0 >2013-12-12 13:36:58,631 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 13:36:58,631 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:36:58,633 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4fc9cd07 >2013-12-12 13:36:58,643 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 441aa385 >2013-12-12 13:36:59,047 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterTasksListVDSCommand, return: [], log id: 4fc9cd07 >2013-12-12 13:36:59,152 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 441aa385 >2013-12-12 13:36:59,154 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3b8cd71b >2013-12-12 13:36:59,349 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6bba8dc2}, log id: 3b8cd71b >2013-12-12 13:37:04,368 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6519d81b >2013-12-12 13:37:04,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6519d81b >2013-12-12 13:37:04,706 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2cf1ff8b >2013-12-12 13:37:04,887 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@72638d2f}, log id: 2cf1ff8b >2013-12-12 13:37:09,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6fef79b5 >2013-12-12 13:37:10,244 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6fef79b5 >2013-12-12 13:37:10,246 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6c82812e >2013-12-12 13:37:10,424 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@469810a6}, log id: 6c82812e >2013-12-12 13:37:15,435 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2084997 >2013-12-12 13:37:15,768 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2084997 >2013-12-12 13:37:15,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5c566c5c >2013-12-12 13:37:15,951 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@883dd548}, log id: 5c566c5c >2013-12-12 13:37:20,965 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 695aeadb >2013-12-12 13:37:21,297 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 695aeadb >2013-12-12 13:37:21,299 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4ba6e129 >2013-12-12 13:37:21,478 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f41eca3e}, log id: 4ba6e129 >2013-12-12 13:37:26,492 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5a64ba81 >2013-12-12 13:37:26,829 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5a64ba81 >2013-12-12 13:37:26,833 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 957964a >2013-12-12 13:37:27,022 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@26d9589f}, log id: 957964a >2013-12-12 13:37:32,033 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1516b1a1 >2013-12-12 13:37:32,375 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1516b1a1 >2013-12-12 13:37:32,377 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 15c1c5c >2013-12-12 13:37:32,557 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a90f7e80}, log id: 15c1c5c >2013-12-12 13:37:37,570 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 13a8b0cc >2013-12-12 13:37:37,904 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 13a8b0cc >2013-12-12 13:37:37,911 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 35906285 >2013-12-12 13:37:38,090 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8784b940}, log id: 35906285 >2013-12-12 13:37:43,107 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2dd8b4e9 >2013-12-12 13:37:43,448 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2dd8b4e9 >2013-12-12 13:37:43,451 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 53f4cbf3 >2013-12-12 13:37:43,632 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d68d62f7}, log id: 53f4cbf3 >2013-12-12 13:37:48,643 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 447e85fd >2013-12-12 13:37:48,990 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 447e85fd >2013-12-12 13:37:48,992 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 453fa613 >2013-12-12 13:37:49,173 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a39836d0}, log id: 453fa613 >2013-12-12 13:37:54,188 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7ec24b2c >2013-12-12 13:37:54,521 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7ec24b2c >2013-12-12 13:37:54,523 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 70312e59 >2013-12-12 13:37:54,699 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@74ec4aab}, log id: 70312e59 >2013-12-12 13:37:59,051 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-98) No up server in cluster >2013-12-12 13:37:59,051 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-98) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:37:59,055 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6fbbc66b >2013-12-12 13:37:59,238 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterTasksListVDSCommand, return: [], log id: 6fbbc66b >2013-12-12 13:37:59,716 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5c6546bf >2013-12-12 13:38:00,046 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5c6546bf >2013-12-12 13:38:00,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5149d016 >2013-12-12 13:38:00,227 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6e08480f}, log id: 5149d016 >2013-12-12 13:38:05,246 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 52f6b43a >2013-12-12 13:38:05,578 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 52f6b43a >2013-12-12 13:38:05,580 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 20f2e0bc >2013-12-12 13:38:05,757 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@79ea2b1f}, log id: 20f2e0bc >2013-12-12 13:38:10,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7eea115f >2013-12-12 13:38:11,111 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7eea115f >2013-12-12 13:38:11,112 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1528f1e4 >2013-12-12 13:38:11,295 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@feddff86}, log id: 1528f1e4 >2013-12-12 13:38:16,314 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 13bf9581 >2013-12-12 13:38:16,645 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 13bf9581 >2013-12-12 13:38:16,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 10862115 >2013-12-12 13:38:16,828 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fcabee59}, log id: 10862115 >2013-12-12 13:38:21,842 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21f7570c >2013-12-12 13:38:22,183 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 21f7570c >2013-12-12 13:38:22,186 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 698f08f0 >2013-12-12 13:38:22,366 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@61d5f1e4}, log id: 698f08f0 >2013-12-12 13:38:27,387 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a65edb0 >2013-12-12 13:38:27,723 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4a65edb0 >2013-12-12 13:38:27,725 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6b4dc7b3 >2013-12-12 13:38:27,904 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b08dbd3b}, log id: 6b4dc7b3 >2013-12-12 13:38:32,919 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1c89f3bd >2013-12-12 13:38:33,255 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1c89f3bd >2013-12-12 13:38:33,256 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 33f49392 >2013-12-12 13:38:33,436 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b39342f5}, log id: 33f49392 >2013-12-12 13:38:38,453 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 795da8bd >2013-12-12 13:38:38,783 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 795da8bd >2013-12-12 13:38:38,785 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5d534bcd >2013-12-12 13:38:38,969 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@89591615}, log id: 5d534bcd >2013-12-12 13:38:43,980 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1f5f701b >2013-12-12 13:38:44,311 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1f5f701b >2013-12-12 13:38:44,313 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 691411ad >2013-12-12 13:38:44,494 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d16bd8ae}, log id: 691411ad >2013-12-12 13:38:49,542 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: bdc637c >2013-12-12 13:38:49,877 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: bdc637c >2013-12-12 13:38:49,879 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3d5adab2 >2013-12-12 13:38:50,057 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@17b71bc6}, log id: 3d5adab2 >2013-12-12 13:38:55,074 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 71515fcb >2013-12-12 13:38:55,409 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 71515fcb >2013-12-12 13:38:55,411 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7fefd72f >2013-12-12 13:38:55,589 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e7dc4a80}, log id: 7fefd72f >2013-12-12 13:38:59,241 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-82) No up server in cluster >2013-12-12 13:38:59,241 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-82) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:38:59,243 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 46e22258 >2013-12-12 13:38:59,428 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterTasksListVDSCommand, return: [], log id: 46e22258 >2013-12-12 13:39:00,607 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 198dadd >2013-12-12 13:39:00,944 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 198dadd >2013-12-12 13:39:00,945 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 50776fd8 >2013-12-12 13:39:01,132 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4b98375a}, log id: 50776fd8 >2013-12-12 13:39:06,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 47394e8e >2013-12-12 13:39:06,470 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 47394e8e >2013-12-12 13:39:06,473 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6901638f >2013-12-12 13:39:06,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6466c81d}, log id: 6901638f >2013-12-12 13:39:11,660 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5dcfe317 >2013-12-12 13:39:11,997 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5dcfe317 >2013-12-12 13:39:12,001 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7a7f248c >2013-12-12 13:39:12,184 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@486a250a}, log id: 7a7f248c >2013-12-12 13:39:17,201 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 437e151a >2013-12-12 13:39:17,542 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 437e151a >2013-12-12 13:39:17,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 11cf58d8 >2013-12-12 13:39:17,717 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@afe6a5c5}, log id: 11cf58d8 >2013-12-12 13:39:22,731 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 30864574 >2013-12-12 13:39:23,079 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 30864574 >2013-12-12 13:39:23,081 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 394a6b66 >2013-12-12 13:39:23,256 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a8155e3b}, log id: 394a6b66 >2013-12-12 13:39:28,292 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 63553f2e >2013-12-12 13:39:28,624 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 63553f2e >2013-12-12 13:39:28,626 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6df14222 >2013-12-12 13:39:28,808 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9dfb1135}, log id: 6df14222 >2013-12-12 13:39:33,825 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1df58c56 >2013-12-12 13:39:34,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1df58c56 >2013-12-12 13:39:34,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 25ffacb4 >2013-12-12 13:39:34,344 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a843f1a9}, log id: 25ffacb4 >2013-12-12 13:39:39,361 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 42b97526 >2013-12-12 13:39:39,706 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 42b97526 >2013-12-12 13:39:39,708 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 551eba18 >2013-12-12 13:39:39,886 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@93b297cf}, log id: 551eba18 >2013-12-12 13:39:44,905 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3007c631 >2013-12-12 13:39:45,241 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3007c631 >2013-12-12 13:39:45,243 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7cb82691 >2013-12-12 13:39:45,427 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c767cd88}, log id: 7cb82691 >2013-12-12 13:39:50,450 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1fb1f672 >2013-12-12 13:39:50,783 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1fb1f672 >2013-12-12 13:39:50,786 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4d08f87c >2013-12-12 13:39:50,963 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@13932ea}, log id: 4d08f87c >2013-12-12 13:39:55,978 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 32bd3c78 >2013-12-12 13:39:56,324 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 32bd3c78 >2013-12-12 13:39:56,325 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7ebffc3e >2013-12-12 13:39:56,501 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dd6b4ccf}, log id: 7ebffc3e >2013-12-12 13:39:59,430 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-65) No up server in cluster >2013-12-12 13:39:59,430 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-65) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:39:59,432 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1bc7012a >2013-12-12 13:39:59,617 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterTasksListVDSCommand, return: [], log id: 1bc7012a >2013-12-12 13:40:01,515 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3060d424 >2013-12-12 13:40:01,859 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3060d424 >2013-12-12 13:40:01,863 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 201a919d >2013-12-12 13:40:02,044 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@336de0ac}, log id: 201a919d >2013-12-12 13:40:07,056 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 12b77cfa >2013-12-12 13:40:07,390 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 12b77cfa >2013-12-12 13:40:07,392 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 59f1a96b >2013-12-12 13:40:07,568 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@960b2308}, log id: 59f1a96b >2013-12-12 13:40:12,579 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 50ef3c1a >2013-12-12 13:40:12,924 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 50ef3c1a >2013-12-12 13:40:12,926 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7420f9b5 >2013-12-12 13:40:13,106 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ff862aaf}, log id: 7420f9b5 >2013-12-12 13:40:18,117 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21aba4f3 >2013-12-12 13:40:18,448 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 21aba4f3 >2013-12-12 13:40:18,452 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21b8eb14 >2013-12-12 13:40:18,630 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fc2d748f}, log id: 21b8eb14 >2013-12-12 13:40:23,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 19acc2fb >2013-12-12 13:40:23,979 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 19acc2fb >2013-12-12 13:40:23,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 28c377f9 >2013-12-12 13:40:24,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@766dbfbd}, log id: 28c377f9 >2013-12-12 13:40:29,172 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5925aa4c >2013-12-12 13:40:29,509 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5925aa4c >2013-12-12 13:40:29,515 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3201f7f2 >2013-12-12 13:40:29,691 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ea312c75}, log id: 3201f7f2 >2013-12-12 13:40:34,701 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2679be85 >2013-12-12 13:40:35,031 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2679be85 >2013-12-12 13:40:35,033 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7dc2de67 >2013-12-12 13:40:35,215 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@406e1b43}, log id: 7dc2de67 >2013-12-12 13:40:40,232 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5eaa12d8 >2013-12-12 13:40:40,564 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5eaa12d8 >2013-12-12 13:40:40,566 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4b09418c >2013-12-12 13:40:40,745 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2bc542ec}, log id: 4b09418c >2013-12-12 13:40:45,756 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 33113170 >2013-12-12 13:40:46,097 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 33113170 >2013-12-12 13:40:46,099 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1898d6e0 >2013-12-12 13:40:46,277 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8e6232ed}, log id: 1898d6e0 >2013-12-12 13:40:51,287 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7a9db03a >2013-12-12 13:40:51,630 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7a9db03a >2013-12-12 13:40:51,632 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 25f2a171 >2013-12-12 13:40:51,808 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@74b33e58}, log id: 25f2a171 >2013-12-12 13:40:56,818 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2e6cf7d1 >2013-12-12 13:40:57,157 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2e6cf7d1 >2013-12-12 13:40:57,159 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 229b2509 >2013-12-12 13:40:57,338 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c4d68201}, log id: 229b2509 >2013-12-12 13:40:58,551 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2a16634b >2013-12-12 13:40:58,738 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@7be8430e, log id: 2a16634b >2013-12-12 13:40:59,619 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-47) No up server in cluster >2013-12-12 13:40:59,620 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-47) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:40:59,622 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 45369723 >2013-12-12 13:40:59,805 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterTasksListVDSCommand, return: [], log id: 45369723 >2013-12-12 13:41:02,388 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5c0f32e7 >2013-12-12 13:41:02,720 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5c0f32e7 >2013-12-12 13:41:02,722 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 69677300 >2013-12-12 13:41:02,902 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ef7d1de6}, log id: 69677300 >2013-12-12 13:41:07,944 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4df1fe20 >2013-12-12 13:41:08,278 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4df1fe20 >2013-12-12 13:41:08,280 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 249ffd1f >2013-12-12 13:41:08,456 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a7ad554}, log id: 249ffd1f >2013-12-12 13:41:13,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1a11625 >2013-12-12 13:41:13,813 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1a11625 >2013-12-12 13:41:13,815 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 19991213 >2013-12-12 13:41:13,994 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ce625a3b}, log id: 19991213 >2013-12-12 13:41:19,008 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 404d2916 >2013-12-12 13:41:19,349 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 404d2916 >2013-12-12 13:41:19,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 37e506cb >2013-12-12 13:41:19,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@62dd838f}, log id: 37e506cb >2013-12-12 13:41:24,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 772d9df7 >2013-12-12 13:41:24,882 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 772d9df7 >2013-12-12 13:41:24,884 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 61d5c335 >2013-12-12 13:41:25,067 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2a8188c2}, log id: 61d5c335 >2013-12-12 13:41:30,077 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 36879ecd >2013-12-12 13:41:30,411 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 36879ecd >2013-12-12 13:41:30,413 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3e91ece4 >2013-12-12 13:41:30,588 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e1e7fe87}, log id: 3e91ece4 >2013-12-12 13:41:35,600 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 391a5c22 >2013-12-12 13:41:35,931 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 391a5c22 >2013-12-12 13:41:35,934 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1f85eb93 >2013-12-12 13:41:36,110 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1d572ed9}, log id: 1f85eb93 >2013-12-12 13:41:41,127 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 63135faa >2013-12-12 13:41:41,465 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 63135faa >2013-12-12 13:41:41,467 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 45d56539 >2013-12-12 13:41:41,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@63390efc}, log id: 45d56539 >2013-12-12 13:41:46,655 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 59ccff3e >2013-12-12 13:41:46,993 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 59ccff3e >2013-12-12 13:41:46,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 604d649d >2013-12-12 13:41:47,176 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b020d5b6}, log id: 604d649d >2013-12-12 13:41:52,192 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5eb41a87 >2013-12-12 13:41:52,522 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5eb41a87 >2013-12-12 13:41:52,524 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 37ebd92b >2013-12-12 13:41:52,703 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@78fd0861}, log id: 37ebd92b >2013-12-12 13:41:57,715 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 75e1ea6c >2013-12-12 13:41:58,050 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 75e1ea6c >2013-12-12 13:41:58,052 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4bfc47d7 >2013-12-12 13:41:58,228 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7c551643}, log id: 4bfc47d7 >2013-12-12 13:41:59,809 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-34) No up server in cluster >2013-12-12 13:41:59,809 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-34) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:41:59,811 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: d2d6e >2013-12-12 13:41:59,991 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterTasksListVDSCommand, return: [], log id: d2d6e >2013-12-12 13:42:03,241 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 17415c3b >2013-12-12 13:42:03,589 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 17415c3b >2013-12-12 13:42:03,591 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 42ebdfc3 >2013-12-12 13:42:03,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9455d6c}, log id: 42ebdfc3 >2013-12-12 13:42:08,780 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 45ea21fe >2013-12-12 13:42:09,111 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 45ea21fe >2013-12-12 13:42:09,113 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a057355 >2013-12-12 13:42:09,290 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4771fb3d}, log id: 4a057355 >2013-12-12 13:42:14,303 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7d50565 >2013-12-12 13:42:14,641 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7d50565 >2013-12-12 13:42:14,643 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3dfbb2a0 >2013-12-12 13:42:14,820 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2a4ba065}, log id: 3dfbb2a0 >2013-12-12 13:42:19,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4acd4aa >2013-12-12 13:42:20,173 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4acd4aa >2013-12-12 13:42:20,175 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7baaa6e5 >2013-12-12 13:42:20,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a32ff292}, log id: 7baaa6e5 >2013-12-12 13:42:25,366 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 308bf113 >2013-12-12 13:42:25,702 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 308bf113 >2013-12-12 13:42:25,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: f3e6c92 >2013-12-12 13:42:25,882 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@20ab08a8}, log id: f3e6c92 >2013-12-12 13:42:30,893 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5f792eb1 >2013-12-12 13:42:31,230 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5f792eb1 >2013-12-12 13:42:31,232 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 25cecf1f >2013-12-12 13:42:31,409 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6f32da17}, log id: 25cecf1f >2013-12-12 13:42:36,420 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3c641015 >2013-12-12 13:42:36,756 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3c641015 >2013-12-12 13:42:36,757 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: bb1c4f0 >2013-12-12 13:42:36,938 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6e890e3f}, log id: bb1c4f0 >2013-12-12 13:42:41,948 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6f2745f9 >2013-12-12 13:42:42,279 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6f2745f9 >2013-12-12 13:42:42,280 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 650e101 >2013-12-12 13:42:42,461 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3b2d0680}, log id: 650e101 >2013-12-12 13:42:47,476 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3e80a35f >2013-12-12 13:42:47,807 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3e80a35f >2013-12-12 13:42:47,809 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5b864e60 >2013-12-12 13:42:47,985 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f58025e8}, log id: 5b864e60 >2013-12-12 13:42:52,996 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 571507f1 >2013-12-12 13:42:53,340 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 571507f1 >2013-12-12 13:42:53,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: ef90646 >2013-12-12 13:42:53,521 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a6d5c9f2}, log id: ef90646 >2013-12-12 13:42:58,530 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6de2bd84 >2013-12-12 13:42:58,863 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6de2bd84 >2013-12-12 13:42:58,865 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 16a318e5 >2013-12-12 13:42:59,041 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ab5b6d57}, log id: 16a318e5 >2013-12-12 13:42:59,994 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-11) No up server in cluster >2013-12-12 13:42:59,994 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-11) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:42:59,996 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 56d3c57a >2013-12-12 13:43:00,175 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterTasksListVDSCommand, return: [], log id: 56d3c57a >2013-12-12 13:43:04,061 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5941f36 >2013-12-12 13:43:04,395 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5941f36 >2013-12-12 13:43:04,397 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1bd9163a >2013-12-12 13:43:04,572 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ab21a57f}, log id: 1bd9163a >2013-12-12 13:43:09,585 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 21691468 >2013-12-12 13:43:09,924 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 21691468 >2013-12-12 13:43:09,926 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 65cc08eb >2013-12-12 13:43:10,103 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c395d94c}, log id: 65cc08eb >2013-12-12 13:43:15,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1c67ec9 >2013-12-12 13:43:15,469 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1c67ec9 >2013-12-12 13:43:15,471 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6d2fa5e7 >2013-12-12 13:43:15,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@90b9cad6}, log id: 6d2fa5e7 >2013-12-12 13:43:20,660 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4dbd39ab >2013-12-12 13:43:20,997 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4dbd39ab >2013-12-12 13:43:20,999 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: e068d1b >2013-12-12 13:43:21,177 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4d775894}, log id: e068d1b >2013-12-12 13:43:26,186 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1af8dac5 >2013-12-12 13:43:26,542 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1af8dac5 >2013-12-12 13:43:26,544 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7ea26ade >2013-12-12 13:43:26,724 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b5a1af7f}, log id: 7ea26ade >2013-12-12 13:43:31,739 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 46717c4e >2013-12-12 13:43:32,073 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 46717c4e >2013-12-12 13:43:32,077 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1be360cc >2013-12-12 13:43:32,254 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1e64f156}, log id: 1be360cc >2013-12-12 13:43:37,267 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3dcba87c >2013-12-12 13:43:37,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3dcba87c >2013-12-12 13:43:37,609 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21d75441 >2013-12-12 13:43:37,784 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd0f201c}, log id: 21d75441 >2013-12-12 13:43:42,797 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 701c6dbd >2013-12-12 13:43:43,134 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 701c6dbd >2013-12-12 13:43:43,137 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3b220c10 >2013-12-12 13:43:43,329 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@25c3d896}, log id: 3b220c10 >2013-12-12 13:43:48,340 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 544fba7a >2013-12-12 13:43:48,682 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 544fba7a >2013-12-12 13:43:48,688 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 66e09009 >2013-12-12 13:43:48,874 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@702a1996}, log id: 66e09009 >2013-12-12 13:43:53,886 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: e3d9eac >2013-12-12 13:43:54,227 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: e3d9eac >2013-12-12 13:43:54,230 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6682f2ad >2013-12-12 13:43:54,408 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4e6a8b8b}, log id: 6682f2ad >2013-12-12 13:43:59,421 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5979449e >2013-12-12 13:43:59,755 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5979449e >2013-12-12 13:43:59,756 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 79aa237b >2013-12-12 13:43:59,931 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@21a23a0b}, log id: 79aa237b >2013-12-12 13:44:00,183 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-96) No up server in cluster >2013-12-12 13:44:00,183 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-96) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:44:00,185 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4eb801c3 >2013-12-12 13:44:00,367 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterTasksListVDSCommand, return: [], log id: 4eb801c3 >2013-12-12 13:44:04,943 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 68ee4ebb >2013-12-12 13:44:05,277 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 68ee4ebb >2013-12-12 13:44:05,279 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7c89c4d >2013-12-12 13:44:05,454 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@73d9484d}, log id: 7c89c4d >2013-12-12 13:44:10,471 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 14c52148 >2013-12-12 13:44:10,808 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 14c52148 >2013-12-12 13:44:10,810 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 315a04de >2013-12-12 13:44:10,985 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f567d585}, log id: 315a04de >2013-12-12 13:44:16,002 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 72a493c2 >2013-12-12 13:44:16,338 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 72a493c2 >2013-12-12 13:44:16,339 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4f3f6170 >2013-12-12 13:44:16,522 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5e3e64c2}, log id: 4f3f6170 >2013-12-12 13:44:21,533 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 571dd755 >2013-12-12 13:44:21,864 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 571dd755 >2013-12-12 13:44:21,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 723877eb >2013-12-12 13:44:22,044 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@14af3176}, log id: 723877eb >2013-12-12 13:44:27,054 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5fd2b3bf >2013-12-12 13:44:27,387 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5fd2b3bf >2013-12-12 13:44:27,389 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 661bb8bd >2013-12-12 13:44:27,564 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f47de817}, log id: 661bb8bd >2013-12-12 13:44:32,575 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 69b808a7 >2013-12-12 13:44:32,907 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 69b808a7 >2013-12-12 13:44:32,909 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 17b7b354 >2013-12-12 13:44:33,084 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6d30f290}, log id: 17b7b354 >2013-12-12 13:44:38,096 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4f12aa1f >2013-12-12 13:44:38,434 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4f12aa1f >2013-12-12 13:44:38,437 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 73232ada >2013-12-12 13:44:38,614 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5d59d2b9}, log id: 73232ada >2013-12-12 13:44:43,624 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5ef16957 >2013-12-12 13:44:43,962 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5ef16957 >2013-12-12 13:44:43,964 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6cc7b2c6 >2013-12-12 13:44:44,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ef7a991e}, log id: 6cc7b2c6 >2013-12-12 13:44:49,151 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 193a3c9f >2013-12-12 13:44:49,502 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 193a3c9f >2013-12-12 13:44:49,503 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 55e72fa5 >2013-12-12 13:44:49,683 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d4b7eb18}, log id: 55e72fa5 >2013-12-12 13:44:54,694 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3d0420c0 >2013-12-12 13:44:55,032 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3d0420c0 >2013-12-12 13:44:55,036 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 8ac12ec >2013-12-12 13:44:55,213 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@387e6b34}, log id: 8ac12ec >2013-12-12 13:45:00,229 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5f2f723a >2013-12-12 13:45:00,370 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-75) No up server in cluster >2013-12-12 13:45:00,371 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-75) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:45:00,373 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 36250c94 >2013-12-12 13:45:00,735 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5f2f723a >2013-12-12 13:45:00,737 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 22f3159b >2013-12-12 13:45:00,744 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterTasksListVDSCommand, return: [], log id: 36250c94 >2013-12-12 13:45:00,916 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@22204781}, log id: 22f3159b >2013-12-12 13:45:05,925 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7351eaec >2013-12-12 13:45:06,258 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7351eaec >2013-12-12 13:45:06,261 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 722ca625 >2013-12-12 13:45:06,436 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@32a1b323}, log id: 722ca625 >2013-12-12 13:45:11,446 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6f5d5ab2 >2013-12-12 13:45:11,784 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6f5d5ab2 >2013-12-12 13:45:11,786 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1548ef37 >2013-12-12 13:45:11,963 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@76356fd}, log id: 1548ef37 >2013-12-12 13:45:16,974 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 22ee00e3 >2013-12-12 13:45:17,317 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 22ee00e3 >2013-12-12 13:45:17,319 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a524b39 >2013-12-12 13:45:17,498 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8f715ad}, log id: 4a524b39 >2013-12-12 13:45:22,514 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 436b511c >2013-12-12 13:45:22,853 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 436b511c >2013-12-12 13:45:22,855 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 95874c5 >2013-12-12 13:45:23,040 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@971127a2}, log id: 95874c5 >2013-12-12 13:45:28,052 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: d4b99ae >2013-12-12 13:45:28,386 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: d4b99ae >2013-12-12 13:45:28,390 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: f5ce485 >2013-12-12 13:45:28,566 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@21c3d9b7}, log id: f5ce485 >2013-12-12 13:45:33,577 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 612432cb >2013-12-12 13:45:33,908 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 612432cb >2013-12-12 13:45:33,914 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: a8693ed >2013-12-12 13:45:34,097 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@178fc62f}, log id: a8693ed >2013-12-12 13:45:39,117 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 678eea0c >2013-12-12 13:45:39,453 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 678eea0c >2013-12-12 13:45:39,458 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 780ef67a >2013-12-12 13:45:39,635 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f6ba17e}, log id: 780ef67a >2013-12-12 13:45:44,645 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 495a56b3 >2013-12-12 13:45:44,992 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 495a56b3 >2013-12-12 13:45:44,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: a3d1f3f >2013-12-12 13:45:45,174 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d1f96b2d}, log id: a3d1f3f >2013-12-12 13:45:50,185 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-41) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2fffd602 >2013-12-12 13:45:50,529 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-41) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2fffd602 >2013-12-12 13:45:50,533 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-41) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: b66317f >2013-12-12 13:45:50,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-41) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6d854868}, log id: b66317f >2013-12-12 13:45:55,724 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 30db4279 >2013-12-12 13:45:56,067 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 30db4279 >2013-12-12 13:45:56,069 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 212b54c3 >2013-12-12 13:45:56,246 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6418435f}, log id: 212b54c3 >2013-12-12 13:45:58,744 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-61) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7fc3e028 >2013-12-12 13:45:58,931 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@76a77b06, log id: 7fc3e028 >2013-12-12 13:46:00,747 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-63) No up server in cluster >2013-12-12 13:46:00,747 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-63) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:46:00,749 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 306de6c6 >2013-12-12 13:46:00,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterTasksListVDSCommand, return: [], log id: 306de6c6 >2013-12-12 13:46:01,256 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1fbf35bc >2013-12-12 13:46:01,599 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1fbf35bc >2013-12-12 13:46:01,601 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 77768b8e >2013-12-12 13:46:01,785 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ebfef9a0}, log id: 77768b8e >2013-12-12 13:46:06,796 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3cd65766 >2013-12-12 13:46:07,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3cd65766 >2013-12-12 13:46:07,143 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4a9e86b4 >2013-12-12 13:46:07,319 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4f07552c}, log id: 4a9e86b4 >2013-12-12 13:46:12,329 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4bc893eb >2013-12-12 13:46:12,674 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4bc893eb >2013-12-12 13:46:12,676 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3405bebd >2013-12-12 13:46:12,855 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@87cedb27}, log id: 3405bebd >2013-12-12 13:46:17,920 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 16fc9001 >2013-12-12 13:46:18,268 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 16fc9001 >2013-12-12 13:46:18,270 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 350b5005 >2013-12-12 13:46:18,450 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bc6c19ee}, log id: 350b5005 >2013-12-12 13:46:23,460 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 266703f1 >2013-12-12 13:46:23,801 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 266703f1 >2013-12-12 13:46:23,803 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5a54ee6e >2013-12-12 13:46:23,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@510b9337}, log id: 5a54ee6e >2013-12-12 13:46:28,992 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 69f4c756 >2013-12-12 13:46:29,327 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 69f4c756 >2013-12-12 13:46:29,328 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 41dea35f >2013-12-12 13:46:29,512 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@86585bd9}, log id: 41dea35f >2013-12-12 13:46:34,521 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1b0dec35 >2013-12-12 13:46:34,857 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1b0dec35 >2013-12-12 13:46:34,858 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: d4efedc >2013-12-12 13:46:35,032 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5679c99b}, log id: d4efedc >2013-12-12 13:46:40,041 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7e6d0a5c >2013-12-12 13:46:40,378 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7e6d0a5c >2013-12-12 13:46:40,380 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6dfbc460 >2013-12-12 13:46:40,554 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@727c1b78}, log id: 6dfbc460 >2013-12-12 13:46:45,564 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7d026304 >2013-12-12 13:46:45,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7d026304 >2013-12-12 13:46:45,901 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 62ac3f84 >2013-12-12 13:46:46,075 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ebb3ede7}, log id: 62ac3f84 >2013-12-12 13:46:51,086 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 797bf340 >2013-12-12 13:46:51,420 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 797bf340 >2013-12-12 13:46:51,421 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: a2114b >2013-12-12 13:46:51,597 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@adbd2d1a}, log id: a2114b >2013-12-12 13:46:56,608 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5f6c19a8 >2013-12-12 13:46:56,943 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5f6c19a8 >2013-12-12 13:46:56,944 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 44bb1d18 >2013-12-12 13:46:57,120 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [64e77997] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd9e3701}, log id: 44bb1d18 >2013-12-12 13:47:00,935 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-46) No up server in cluster >2013-12-12 13:47:00,935 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-46) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:47:00,938 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2e834e50 >2013-12-12 13:47:01,120 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterTasksListVDSCommand, return: [], log id: 2e834e50 >2013-12-12 13:47:02,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: dac944a >2013-12-12 13:47:02,497 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: dac944a >2013-12-12 13:47:02,499 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4169b9b2 >2013-12-12 13:47:02,678 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@942f9253}, log id: 4169b9b2 >2013-12-12 13:47:07,693 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3bf9dbba >2013-12-12 13:47:08,025 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3bf9dbba >2013-12-12 13:47:08,027 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 62cd235e >2013-12-12 13:47:08,207 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7719842e}, log id: 62cd235e >2013-12-12 13:47:13,218 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1499fee4 >2013-12-12 13:47:13,555 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1499fee4 >2013-12-12 13:47:13,557 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3b5c7931 >2013-12-12 13:47:13,732 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@22a9dad9}, log id: 3b5c7931 >2013-12-12 13:47:18,742 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5404396 >2013-12-12 13:47:19,079 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5404396 >2013-12-12 13:47:19,081 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 20632577 >2013-12-12 13:47:19,258 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8a634f62}, log id: 20632577 >2013-12-12 13:47:24,269 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3688ad01 >2013-12-12 13:47:24,605 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3688ad01 >2013-12-12 13:47:24,607 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 86cff17 >2013-12-12 13:47:24,784 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2d7a3e1b}, log id: 86cff17 >2013-12-12 13:47:27,470 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-2) Theme /80-rhs-1.brand has no property defined for key resources >2013-12-12 13:47:29,795 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 675207ac >2013-12-12 13:47:30,127 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 675207ac >2013-12-12 13:47:30,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 27c74d67 >2013-12-12 13:47:30,307 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@976b78af}, log id: 27c74d67 >2013-12-12 13:47:35,316 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 721f6fb >2013-12-12 13:47:35,649 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 721f6fb >2013-12-12 13:47:35,651 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 24f6436 >2013-12-12 13:47:35,831 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5d2071e1}, log id: 24f6436 >2013-12-12 13:47:39,836 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-7) Theme /80-rhs-1.brand has no property defined for key resources >2013-12-12 13:47:40,842 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 253e663 >2013-12-12 13:47:41,178 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 253e663 >2013-12-12 13:47:41,180 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 216ed38a >2013-12-12 13:47:41,357 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e0411a9c}, log id: 216ed38a >2013-12-12 13:47:46,373 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1bb38217 >2013-12-12 13:47:46,708 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1bb38217 >2013-12-12 13:47:46,709 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5557262c >2013-12-12 13:47:46,884 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4cd2a2c1}, log id: 5557262c >2013-12-12 13:47:48,991 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-6) Theme /80-rhs-1.brand has no property defined for key resources >2013-12-12 13:47:51,196 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginAdminUserCommand internal: false. >2013-12-12 13:47:51,294 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 13:47:51,899 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1f6031db >2013-12-12 13:47:52,236 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1f6031db >2013-12-12 13:47:52,237 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 586cefa4 >2013-12-12 13:47:52,413 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5889b199}, log id: 586cefa4 >2013-12-12 13:47:57,424 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6058c042 >2013-12-12 13:47:57,761 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6058c042 >2013-12-12 13:47:57,763 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1008d38b >2013-12-12 13:47:57,941 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@10326072}, log id: 1008d38b >2013-12-12 13:48:01,122 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-29) No up server in cluster >2013-12-12 13:48:01,122 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-29) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:48:01,124 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7d6f2845 >2013-12-12 13:48:01,301 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7d6f2845 >2013-12-12 13:48:02,952 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 30a39506 >2013-12-12 13:48:03,291 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 30a39506 >2013-12-12 13:48:03,293 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1ed65293 >2013-12-12 13:48:03,473 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3309cbbd}, log id: 1ed65293 >2013-12-12 13:48:08,486 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6a1cc77 >2013-12-12 13:48:08,827 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6a1cc77 >2013-12-12 13:48:08,828 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2fcf183a >2013-12-12 13:48:09,005 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@99826ac0}, log id: 2fcf183a >2013-12-12 13:48:14,021 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: e85624f >2013-12-12 13:48:14,356 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: e85624f >2013-12-12 13:48:14,358 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 34d66a8b >2013-12-12 13:48:14,537 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d484bf3c}, log id: 34d66a8b >2013-12-12 13:48:19,546 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 37223aa2 >2013-12-12 13:48:19,889 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 37223aa2 >2013-12-12 13:48:19,891 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4f432ed7 >2013-12-12 13:48:20,069 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@65a3bcc7}, log id: 4f432ed7 >2013-12-12 13:48:25,079 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 710fffb9 >2013-12-12 13:48:25,423 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 710fffb9 >2013-12-12 13:48:25,425 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6291f197 >2013-12-12 13:48:25,601 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6f4f6d2e}, log id: 6291f197 >2013-12-12 13:48:30,610 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 794bf19e >2013-12-12 13:48:30,950 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 794bf19e >2013-12-12 13:48:30,952 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 166124ca >2013-12-12 13:48:31,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cf78e66}, log id: 166124ca >2013-12-12 13:48:36,140 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1f9a3899 >2013-12-12 13:48:36,476 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1f9a3899 >2013-12-12 13:48:36,477 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 68d3494b >2013-12-12 13:48:36,656 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@abfc89d5}, log id: 68d3494b >2013-12-12 13:48:41,668 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: db2ab1e >2013-12-12 13:48:42,010 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: db2ab1e >2013-12-12 13:48:42,011 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 76f987d >2013-12-12 13:48:42,188 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bd35595b}, log id: 76f987d >2013-12-12 13:48:47,207 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 352a12ae >2013-12-12 13:48:47,539 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 352a12ae >2013-12-12 13:48:47,540 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 73e4739 >2013-12-12 13:48:47,717 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@722934c4}, log id: 73e4739 >2013-12-12 13:48:52,728 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5d61dcc8 >2013-12-12 13:48:53,065 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5d61dcc8 >2013-12-12 13:48:53,066 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4b1d7aaf >2013-12-12 13:48:53,250 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@23a39fc3}, log id: 4b1d7aaf >2013-12-12 13:48:56,811 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-10) Theme /80-rhs-1.brand has no property defined for key resources >2013-12-12 13:48:58,261 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4a28a462 >2013-12-12 13:48:58,596 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4a28a462 >2013-12-12 13:48:58,598 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: cb68d0f >2013-12-12 13:48:58,776 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5b578886}, log id: cb68d0f >2013-12-12 13:49:01,304 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-2) No up server in cluster >2013-12-12 13:49:01,304 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-2) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:49:01,307 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-2) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5598bb4d >2013-12-12 13:49:01,487 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5598bb4d >2013-12-12 13:49:03,803 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 21557e34 >2013-12-12 13:49:04,139 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 21557e34 >2013-12-12 13:49:04,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7f60e4f4 >2013-12-12 13:49:04,320 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@466d790d}, log id: 7f60e4f4 >2013-12-12 13:49:09,335 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1dc6fedf >2013-12-12 13:49:09,667 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1dc6fedf >2013-12-12 13:49:09,669 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1c874dfd >2013-12-12 13:49:09,848 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@23b94f51}, log id: 1c874dfd >2013-12-12 13:49:14,858 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3c6c11dc >2013-12-12 13:49:15,196 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3c6c11dc >2013-12-12 13:49:15,198 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3d50d7d4 >2013-12-12 13:49:15,373 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@64392136}, log id: 3d50d7d4 >2013-12-12 13:49:20,387 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7ad12780 >2013-12-12 13:49:20,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7ad12780 >2013-12-12 13:49:20,731 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7652b454 >2013-12-12 13:49:20,909 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6022cea}, log id: 7652b454 >2013-12-12 13:49:25,920 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 535f4da3 >2013-12-12 13:49:26,251 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 535f4da3 >2013-12-12 13:49:26,253 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 59edcfac >2013-12-12 13:49:26,428 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8eebdec8}, log id: 59edcfac >2013-12-12 13:49:31,438 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 56262e0a >2013-12-12 13:49:31,777 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 56262e0a >2013-12-12 13:49:31,778 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: c471bd0 >2013-12-12 13:49:31,959 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@251ff585}, log id: c471bd0 >2013-12-12 13:49:36,972 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3856bcee >2013-12-12 13:49:37,306 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3856bcee >2013-12-12 13:49:37,307 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 53bca3b5 >2013-12-12 13:49:37,484 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@df8c63a7}, log id: 53bca3b5 >2013-12-12 13:49:42,494 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6269fec6 >2013-12-12 13:49:42,829 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6269fec6 >2013-12-12 13:49:42,831 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6701d266 >2013-12-12 13:49:43,007 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@efb59b7b}, log id: 6701d266 >2013-12-12 13:49:48,018 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 69b158ee >2013-12-12 13:49:48,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 69b158ee >2013-12-12 13:49:48,357 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 45b38a16 >2013-12-12 13:49:48,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@83d351f8}, log id: 45b38a16 >2013-12-12 13:49:53,542 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2bf36c8e >2013-12-12 13:49:53,877 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2bf36c8e >2013-12-12 13:49:53,879 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2d38f12f >2013-12-12 13:49:54,057 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2f63782f}, log id: 2d38f12f >2013-12-12 13:49:59,070 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 63aaaed2 >2013-12-12 13:49:59,408 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 63aaaed2 >2013-12-12 13:49:59,410 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4391456d >2013-12-12 13:49:59,587 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bc9c4396}, log id: 4391456d >2013-12-12 13:50:01,491 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-94) No up server in cluster >2013-12-12 13:50:01,491 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-94) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:50:01,493 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 330db854 >2013-12-12 13:50:01,673 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterTasksListVDSCommand, return: [], log id: 330db854 >2013-12-12 13:50:04,614 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 161c9a7a >2013-12-12 13:50:04,950 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 161c9a7a >2013-12-12 13:50:04,952 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 59d74bb3 >2013-12-12 13:50:05,129 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a004dac0}, log id: 59d74bb3 >2013-12-12 13:50:10,139 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3e3380d2 >2013-12-12 13:50:10,470 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3e3380d2 >2013-12-12 13:50:10,472 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 622c9d01 >2013-12-12 13:50:10,648 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@97a9a220}, log id: 622c9d01 >2013-12-12 13:50:15,659 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 783680cf >2013-12-12 13:50:15,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 783680cf >2013-12-12 13:50:15,997 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3060afe3 >2013-12-12 13:50:16,180 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f906f3d4}, log id: 3060afe3 >2013-12-12 13:50:21,190 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 67d40048 >2013-12-12 13:50:21,522 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 67d40048 >2013-12-12 13:50:21,524 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 11f15909 >2013-12-12 13:50:21,699 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b4cac7ea}, log id: 11f15909 >2013-12-12 13:50:26,710 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6c7d3e04 >2013-12-12 13:50:27,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6c7d3e04 >2013-12-12 13:50:27,052 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 37819ee4 >2013-12-12 13:50:27,228 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5c833048}, log id: 37819ee4 >2013-12-12 13:50:32,239 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 53ef55dd >2013-12-12 13:50:32,582 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 53ef55dd >2013-12-12 13:50:32,583 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 19fe5159 >2013-12-12 13:50:32,762 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e0e093c8}, log id: 19fe5159 >2013-12-12 13:50:37,775 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 44f27953 >2013-12-12 13:50:38,106 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 44f27953 >2013-12-12 13:50:38,108 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5324c3a2 >2013-12-12 13:50:38,284 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@19208f56}, log id: 5324c3a2 >2013-12-12 13:50:43,295 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5e6e4109 >2013-12-12 13:50:43,634 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5e6e4109 >2013-12-12 13:50:43,635 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: d9b2a92 >2013-12-12 13:50:43,811 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3cf75cfb}, log id: d9b2a92 >2013-12-12 13:50:48,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5b4196a >2013-12-12 13:50:49,163 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5b4196a >2013-12-12 13:50:49,164 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1d3880d0 >2013-12-12 13:50:49,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2caf65af}, log id: 1d3880d0 >2013-12-12 13:50:54,352 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 580a460b >2013-12-12 13:50:54,685 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 580a460b >2013-12-12 13:50:54,687 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 57b160ff >2013-12-12 13:50:54,864 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@23144a92}, log id: 57b160ff >2013-12-12 13:50:58,937 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 669408fd >2013-12-12 13:50:59,125 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@108a3522, log id: 669408fd >2013-12-12 13:50:59,875 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6487d161 >2013-12-12 13:51:00,211 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6487d161 >2013-12-12 13:51:00,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 558fab00 >2013-12-12 13:51:00,386 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3bd3494b}, log id: 558fab00 >2013-12-12 13:51:01,678 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-74) No up server in cluster >2013-12-12 13:51:01,678 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-74) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:51:01,681 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 75eda028 >2013-12-12 13:51:01,860 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterTasksListVDSCommand, return: [], log id: 75eda028 >2013-12-12 13:51:05,396 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 13d7acac >2013-12-12 13:51:05,735 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 13d7acac >2013-12-12 13:51:05,737 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2ab5069e >2013-12-12 13:51:05,916 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d7052a87}, log id: 2ab5069e >2013-12-12 13:51:10,927 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2462a94d >2013-12-12 13:51:11,261 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2462a94d >2013-12-12 13:51:11,262 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6ac6edd >2013-12-12 13:51:11,439 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@118ab59a}, log id: 6ac6edd >2013-12-12 13:51:16,450 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 47e80fd3 >2013-12-12 13:51:16,788 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 47e80fd3 >2013-12-12 13:51:16,790 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3c6df0ec >2013-12-12 13:51:16,966 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c352df98}, log id: 3c6df0ec >2013-12-12 13:51:21,976 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 10b869f0 >2013-12-12 13:51:22,319 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 10b869f0 >2013-12-12 13:51:22,321 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 606e8af0 >2013-12-12 13:51:22,497 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f86c3d89}, log id: 606e8af0 >2013-12-12 13:51:27,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 717dcb1b >2013-12-12 13:51:27,840 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 717dcb1b >2013-12-12 13:51:27,844 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2e62e2ef >2013-12-12 13:51:28,022 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5bf57e75}, log id: 2e62e2ef >2013-12-12 13:51:33,039 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4d064d26 >2013-12-12 13:51:33,372 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4d064d26 >2013-12-12 13:51:33,374 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7e5a6100 >2013-12-12 13:51:33,549 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@21aa9084}, log id: 7e5a6100 >2013-12-12 13:51:38,560 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4b97afe6 >2013-12-12 13:51:38,894 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4b97afe6 >2013-12-12 13:51:38,896 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 31a60c58 >2013-12-12 13:51:39,085 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8b3e91ae}, log id: 31a60c58 >2013-12-12 13:51:44,097 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 25422be2 >2013-12-12 13:51:44,432 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 25422be2 >2013-12-12 13:51:44,434 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 49a0024d >2013-12-12 13:51:44,611 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-25) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2811b1ec}, log id: 49a0024d >2013-12-12 13:51:49,622 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 35f27da0 >2013-12-12 13:51:49,961 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 35f27da0 >2013-12-12 13:51:49,962 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 10d2da6c >2013-12-12 13:51:50,141 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1e9c1242}, log id: 10d2da6c >2013-12-12 13:51:55,152 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 795c45e8 >2013-12-12 13:51:55,485 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 795c45e8 >2013-12-12 13:51:55,487 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3e7f214 >2013-12-12 13:51:55,663 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c928cb88}, log id: 3e7f214 >2013-12-12 13:52:00,674 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 683cd887 >2013-12-12 13:52:01,014 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 683cd887 >2013-12-12 13:52:01,016 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 26995f5f >2013-12-12 13:52:01,191 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7937db1c}, log id: 26995f5f >2013-12-12 13:52:01,863 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-57) No up server in cluster >2013-12-12 13:52:01,863 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-57) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:52:01,865 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7f10fd1c >2013-12-12 13:52:02,048 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7f10fd1c >2013-12-12 13:52:06,201 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3f815089 >2013-12-12 13:52:06,541 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3f815089 >2013-12-12 13:52:06,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2146f5e6 >2013-12-12 13:52:06,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@946278b0}, log id: 2146f5e6 >2013-12-12 13:52:11,730 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1f03d2ef >2013-12-12 13:52:12,065 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1f03d2ef >2013-12-12 13:52:12,067 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1176abfc >2013-12-12 13:52:12,249 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@77ee318d}, log id: 1176abfc >2013-12-12 13:52:17,259 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2b0302ec >2013-12-12 13:52:17,593 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2b0302ec >2013-12-12 13:52:17,595 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 37577b01 >2013-12-12 13:52:17,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f72dbe5}, log id: 37577b01 >2013-12-12 13:52:22,781 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6354c1d3 >2013-12-12 13:52:23,125 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6354c1d3 >2013-12-12 13:52:23,128 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5089ac98 >2013-12-12 13:52:23,311 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c49f5dcc}, log id: 5089ac98 >2013-12-12 13:52:28,322 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 649c37ba >2013-12-12 13:52:28,656 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 649c37ba >2013-12-12 13:52:28,658 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 178613f3 >2013-12-12 13:52:28,837 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@701e1bc5}, log id: 178613f3 >2013-12-12 13:52:33,847 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 36f4717b >2013-12-12 13:52:34,187 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 36f4717b >2013-12-12 13:52:34,188 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 258867c5 >2013-12-12 13:52:34,361 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e2d04ab0}, log id: 258867c5 >2013-12-12 13:52:39,371 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6ccf858a >2013-12-12 13:52:39,705 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6ccf858a >2013-12-12 13:52:39,707 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 109dae34 >2013-12-12 13:52:39,884 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2ee13692}, log id: 109dae34 >2013-12-12 13:52:44,894 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 68481f29 >2013-12-12 13:52:45,231 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 68481f29 >2013-12-12 13:52:45,232 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 66cbe895 >2013-12-12 13:52:45,416 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@89be9e7d}, log id: 66cbe895 >2013-12-12 13:52:50,431 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 439cc905 >2013-12-12 13:52:50,781 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 439cc905 >2013-12-12 13:52:50,782 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 22b2f08b >2013-12-12 13:52:50,970 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c0340d7c}, log id: 22b2f08b >2013-12-12 13:52:55,983 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: ac83b9f >2013-12-12 13:52:56,323 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: ac83b9f >2013-12-12 13:52:56,324 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 418632a >2013-12-12 13:52:56,502 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b27e9180}, log id: 418632a >2013-12-12 13:53:01,512 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7e761212 >2013-12-12 13:53:01,851 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7e761212 >2013-12-12 13:53:01,853 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 498787be >2013-12-12 13:53:02,035 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b7e0df1c}, log id: 498787be >2013-12-12 13:53:02,051 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-32) No up server in cluster >2013-12-12 13:53:02,051 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-32) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:53:02,054 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5955297f >2013-12-12 13:53:02,235 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5955297f >2013-12-12 13:53:07,046 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2c41ddd7 >2013-12-12 13:53:07,380 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2c41ddd7 >2013-12-12 13:53:07,382 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 11c7644a >2013-12-12 13:53:07,562 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3aa7362c}, log id: 11c7644a >2013-12-12 13:53:12,571 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 50db733c >2013-12-12 13:53:12,909 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 50db733c >2013-12-12 13:53:12,911 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 41d23a11 >2013-12-12 13:53:13,090 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@51196079}, log id: 41d23a11 >2013-12-12 13:53:18,100 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 23552ad9 >2013-12-12 13:53:18,435 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 23552ad9 >2013-12-12 13:53:18,438 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 48771a2 >2013-12-12 13:53:18,627 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a8b4c185}, log id: 48771a2 >2013-12-12 13:53:23,637 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 36a14643 >2013-12-12 13:53:23,976 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 36a14643 >2013-12-12 13:53:23,977 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 652df09e >2013-12-12 13:53:24,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8843b1cd}, log id: 652df09e >2013-12-12 13:53:29,169 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2a141db1 >2013-12-12 13:53:29,506 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2a141db1 >2013-12-12 13:53:29,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 684003b1 >2013-12-12 13:53:29,683 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3a951531}, log id: 684003b1 >2013-12-12 13:53:34,693 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 31cf6074 >2013-12-12 13:53:35,029 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 31cf6074 >2013-12-12 13:53:35,030 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1dc071f0 >2013-12-12 13:53:35,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@482b6833}, log id: 1dc071f0 >2013-12-12 13:53:40,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 33caf1ce >2013-12-12 13:53:40,554 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 33caf1ce >2013-12-12 13:53:40,558 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 9a2b781 >2013-12-12 13:53:40,734 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@103d7b8d}, log id: 9a2b781 >2013-12-12 13:53:45,747 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3aeac42c >2013-12-12 13:53:46,080 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3aeac42c >2013-12-12 13:53:46,082 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3aef7263 >2013-12-12 13:53:46,259 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@287655ab}, log id: 3aef7263 >2013-12-12 13:53:51,289 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 796ecb06 >2013-12-12 13:53:51,623 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 796ecb06 >2013-12-12 13:53:51,624 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: cd7c397 >2013-12-12 13:53:51,801 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@966bacc5}, log id: cd7c397 >2013-12-12 13:53:56,811 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6ce3b37d >2013-12-12 13:53:57,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6ce3b37d >2013-12-12 13:53:57,164 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 68c6fcf6 >2013-12-12 13:53:57,343 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e9b5e2b0}, log id: 68c6fcf6 >2013-12-12 13:54:02,238 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-21) No up server in cluster >2013-12-12 13:54:02,238 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-21) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:54:02,240 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 70dca8bc >2013-12-12 13:54:02,358 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 44c5496f >2013-12-12 13:54:02,473 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterTasksListVDSCommand, return: [], log id: 70dca8bc >2013-12-12 13:54:02,698 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 44c5496f >2013-12-12 13:54:02,700 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 475ed79 >2013-12-12 13:54:02,875 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f7164aef}, log id: 475ed79 >2013-12-12 13:54:07,887 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7fe97bdb >2013-12-12 13:54:08,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7fe97bdb >2013-12-12 13:54:08,224 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 753e7e82 >2013-12-12 13:54:08,401 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e302c3e0}, log id: 753e7e82 >2013-12-12 13:54:13,410 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7462741c >2013-12-12 13:54:13,743 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7462741c >2013-12-12 13:54:13,749 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 38e39fdd >2013-12-12 13:54:13,927 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@261c6907}, log id: 38e39fdd >2013-12-12 13:54:18,937 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5eca52ac >2013-12-12 13:54:19,281 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5eca52ac >2013-12-12 13:54:19,283 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7225136 >2013-12-12 13:54:19,459 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2e26deec}, log id: 7225136 >2013-12-12 13:54:24,475 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7c80ec55 >2013-12-12 13:54:24,808 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7c80ec55 >2013-12-12 13:54:24,809 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 544bcea6 >2013-12-12 13:54:24,990 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f9585b8a}, log id: 544bcea6 >2013-12-12 13:54:30,000 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 73f9b541 >2013-12-12 13:54:30,333 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 73f9b541 >2013-12-12 13:54:30,334 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 16591800 >2013-12-12 13:54:30,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@63409e85}, log id: 16591800 >2013-12-12 13:54:35,526 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 11d3db32 >2013-12-12 13:54:35,855 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 11d3db32 >2013-12-12 13:54:35,857 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 66d783dc >2013-12-12 13:54:36,034 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@10b5dc0f}, log id: 66d783dc >2013-12-12 13:54:41,045 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7f38db23 >2013-12-12 13:54:41,377 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7f38db23 >2013-12-12 13:54:41,380 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 45dc5661 >2013-12-12 13:54:41,556 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@89a8e674}, log id: 45dc5661 >2013-12-12 13:54:46,566 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 26469263 >2013-12-12 13:54:46,900 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 26469263 >2013-12-12 13:54:46,902 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: db63b94 >2013-12-12 13:54:47,079 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dcb9d260}, log id: db63b94 >2013-12-12 13:54:52,090 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1a839b25 >2013-12-12 13:54:52,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1a839b25 >2013-12-12 13:54:52,431 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 14bd2107 >2013-12-12 13:54:52,609 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a0697010}, log id: 14bd2107 >2013-12-12 13:54:57,621 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4cc5aadc >2013-12-12 13:54:57,957 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4cc5aadc >2013-12-12 13:54:57,958 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2ed1f8dc >2013-12-12 13:54:58,135 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@18ba1dda}, log id: 2ed1f8dc >2013-12-12 13:55:02,476 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-4) No up server in cluster >2013-12-12 13:55:02,476 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-4) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:55:02,479 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5775309c >2013-12-12 13:55:02,658 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5775309c >2013-12-12 13:55:03,146 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3deba927 >2013-12-12 13:55:03,478 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3deba927 >2013-12-12 13:55:03,479 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 364866ae >2013-12-12 13:55:03,655 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@188d3441}, log id: 364866ae >2013-12-12 13:55:08,664 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 39cf6db2 >2013-12-12 13:55:08,998 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 39cf6db2 >2013-12-12 13:55:09,000 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7367ef08 >2013-12-12 13:55:09,175 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a1094783}, log id: 7367ef08 >2013-12-12 13:55:14,185 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7507e9c0 >2013-12-12 13:55:14,518 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7507e9c0 >2013-12-12 13:55:14,519 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 68130b7e >2013-12-12 13:55:14,697 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c9f28991}, log id: 68130b7e >2013-12-12 13:55:19,718 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 384ee001 >2013-12-12 13:55:20,053 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 384ee001 >2013-12-12 13:55:20,055 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 577edf81 >2013-12-12 13:55:20,235 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8830f014}, log id: 577edf81 >2013-12-12 13:55:25,248 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 144faf83 >2013-12-12 13:55:25,588 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 144faf83 >2013-12-12 13:55:25,589 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5c8785db >2013-12-12 13:55:25,765 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d2d7929b}, log id: 5c8785db >2013-12-12 13:55:30,776 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3d1d383d >2013-12-12 13:55:31,115 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3d1d383d >2013-12-12 13:55:31,116 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 71de581f >2013-12-12 13:55:31,292 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@af58b5b1}, log id: 71de581f >2013-12-12 13:55:36,303 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 65de24f9 >2013-12-12 13:55:36,633 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 65de24f9 >2013-12-12 13:55:36,635 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6a201150 >2013-12-12 13:55:36,809 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dc91fbc4}, log id: 6a201150 >2013-12-12 13:55:41,820 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 273a233 >2013-12-12 13:55:42,161 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 273a233 >2013-12-12 13:55:42,163 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2b473a5f >2013-12-12 13:55:42,342 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ba7adf99}, log id: 2b473a5f >2013-12-12 13:55:47,352 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 77ff0dc0 >2013-12-12 13:55:47,688 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 77ff0dc0 >2013-12-12 13:55:47,690 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 16e43e13 >2013-12-12 13:55:47,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ebe28ce2}, log id: 16e43e13 >2013-12-12 13:55:52,875 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: c0b7300 >2013-12-12 13:55:53,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: c0b7300 >2013-12-12 13:55:53,214 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 47faafd >2013-12-12 13:55:53,390 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@76a18570}, log id: 47faafd >2013-12-12 13:55:58,404 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: d80b841 >2013-12-12 13:55:58,739 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: d80b841 >2013-12-12 13:55:58,741 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 646644cb >2013-12-12 13:55:58,921 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5e5f64db}, log id: 646644cb >2013-12-12 13:55:59,131 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 16ea6338 >2013-12-12 13:55:59,314 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@66b7deb1, log id: 16ea6338 >2013-12-12 13:56:02,661 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-88) No up server in cluster >2013-12-12 13:56:02,661 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-88) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:56:02,664 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5bbd20a0 >2013-12-12 13:56:02,842 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GlusterTasksListVDSCommand, return: [], log id: 5bbd20a0 >2013-12-12 13:56:03,931 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 47f1c5c9 >2013-12-12 13:56:04,266 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 47f1c5c9 >2013-12-12 13:56:04,268 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3c813b66 >2013-12-12 13:56:04,452 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@498387e3}, log id: 3c813b66 >2013-12-12 13:56:09,463 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 161385c3 >2013-12-12 13:56:09,795 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 161385c3 >2013-12-12 13:56:09,796 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-99) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5c84b7c3 >2013-12-12 13:56:09,971 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-99) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f505ba99}, log id: 5c84b7c3 >2013-12-12 13:56:14,981 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 359665bb >2013-12-12 13:56:15,317 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 359665bb >2013-12-12 13:56:15,319 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6f815132 >2013-12-12 13:56:15,496 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6715dff9}, log id: 6f815132 >2013-12-12 13:56:20,505 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 18d0b3cd >2013-12-12 13:56:20,847 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 18d0b3cd >2013-12-12 13:56:20,849 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4b1b1c0d >2013-12-12 13:56:21,028 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-14) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8dc3053d}, log id: 4b1b1c0d >2013-12-12 13:56:26,038 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7545cbd2 >2013-12-12 13:56:26,368 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7545cbd2 >2013-12-12 13:56:26,370 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7b52bf05 >2013-12-12 13:56:26,545 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@15145670}, log id: 7b52bf05 >2013-12-12 13:56:31,556 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4b884dbf >2013-12-12 13:56:31,888 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4b884dbf >2013-12-12 13:56:31,890 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 372bdfe8 >2013-12-12 13:56:32,071 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ac5f479a}, log id: 372bdfe8 >2013-12-12 13:56:37,081 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 13d882e >2013-12-12 13:56:37,421 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 13d882e >2013-12-12 13:56:37,422 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 76437951 >2013-12-12 13:56:37,603 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c163293d}, log id: 76437951 >2013-12-12 13:56:42,613 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7dcda129 >2013-12-12 13:56:42,945 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7dcda129 >2013-12-12 13:56:42,947 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 50387cda >2013-12-12 13:56:43,126 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@643cb3fd}, log id: 50387cda >2013-12-12 13:56:48,137 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 23dda386 >2013-12-12 13:56:48,472 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 23dda386 >2013-12-12 13:56:48,474 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7d4c3035 >2013-12-12 13:56:48,650 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ac084087}, log id: 7d4c3035 >2013-12-12 13:56:53,664 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3fdacd60 >2013-12-12 13:56:53,995 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3fdacd60 >2013-12-12 13:56:53,998 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 38e9f >2013-12-12 13:56:54,175 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@143479fc}, log id: 38e9f >2013-12-12 13:56:59,187 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 19903bd3 >2013-12-12 13:56:59,525 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 19903bd3 >2013-12-12 13:56:59,527 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 58ea0e43 >2013-12-12 13:56:59,704 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bdaadb7b}, log id: 58ea0e43 >2013-12-12 13:57:02,844 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-71) No up server in cluster >2013-12-12 13:57:02,844 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-71) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:57:02,849 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-71) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 71370fdf >2013-12-12 13:57:03,030 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-71) FINISH, GlusterTasksListVDSCommand, return: [], log id: 71370fdf >2013-12-12 13:57:04,714 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4f89b188 >2013-12-12 13:57:05,049 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4f89b188 >2013-12-12 13:57:05,051 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 50a8740c >2013-12-12 13:57:05,229 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@eaaa0b6b}, log id: 50a8740c >2013-12-12 13:57:10,240 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5467f142 >2013-12-12 13:57:10,578 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5467f142 >2013-12-12 13:57:10,579 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1dfbce3b >2013-12-12 13:57:10,757 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cc623ed4}, log id: 1dfbce3b >2013-12-12 13:57:15,767 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 14598a9e >2013-12-12 13:57:16,102 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 14598a9e >2013-12-12 13:57:16,105 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 57567bcd >2013-12-12 13:57:16,284 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@92fb3c46}, log id: 57567bcd >2013-12-12 13:57:21,297 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4867a190 >2013-12-12 13:57:21,634 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4867a190 >2013-12-12 13:57:21,635 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 199ef7ba >2013-12-12 13:57:21,813 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@de4f1dc3}, log id: 199ef7ba >2013-12-12 13:57:26,825 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1db2b8a >2013-12-12 13:57:27,172 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1db2b8a >2013-12-12 13:57:27,174 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7be63dc >2013-12-12 13:57:27,351 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8f0ea242}, log id: 7be63dc >2013-12-12 13:57:32,362 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 43afb15a >2013-12-12 13:57:32,701 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 43afb15a >2013-12-12 13:57:32,703 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2ba024dc >2013-12-12 13:57:32,883 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2c282b89}, log id: 2ba024dc >2013-12-12 13:57:37,917 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 39c969d7 >2013-12-12 13:57:38,250 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 39c969d7 >2013-12-12 13:57:38,252 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6cb6cff0 >2013-12-12 13:57:38,429 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@be77139b}, log id: 6cb6cff0 >2013-12-12 13:57:43,439 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3a85fc3f >2013-12-12 13:57:43,777 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3a85fc3f >2013-12-12 13:57:43,780 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5dae70c7 >2013-12-12 13:57:43,960 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f041027a}, log id: 5dae70c7 >2013-12-12 13:57:48,970 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3cc554cd >2013-12-12 13:57:49,299 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 3cc554cd >2013-12-12 13:57:49,306 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1bdace8c >2013-12-12 13:57:49,483 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@53d0e8fc}, log id: 1bdace8c >2013-12-12 13:57:54,496 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 287d8804 >2013-12-12 13:57:54,832 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 287d8804 >2013-12-12 13:57:54,834 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 484a8fda >2013-12-12 13:57:55,013 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-32) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@56ec8ca}, log id: 484a8fda >2013-12-12 13:58:00,027 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 15892311 >2013-12-12 13:58:00,365 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 15892311 >2013-12-12 13:58:00,367 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1068c1ca >2013-12-12 13:58:00,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e31633cd}, log id: 1068c1ca >2013-12-12 13:58:03,033 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-50) No up server in cluster >2013-12-12 13:58:03,033 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-50) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:58:03,036 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7d72e8f3 >2013-12-12 13:58:03,216 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7d72e8f3 >2013-12-12 13:58:05,553 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 66645c1d >2013-12-12 13:58:05,896 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 66645c1d >2013-12-12 13:58:05,898 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 23b36878 >2013-12-12 13:58:06,077 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-58) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8dc0aa83}, log id: 23b36878 >2013-12-12 13:58:11,091 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 55dd7dc7 >2013-12-12 13:58:11,423 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 55dd7dc7 >2013-12-12 13:58:11,425 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2e53e276 >2013-12-12 13:58:11,600 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-60) [44ea9dad] FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@673ac5b0}, log id: 2e53e276 >2013-12-12 13:58:16,614 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 364daa51 >2013-12-12 13:58:16,949 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 364daa51 >2013-12-12 13:58:16,950 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1ab480a >2013-12-12 13:58:17,131 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5273e6d9}, log id: 1ab480a >2013-12-12 13:58:22,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7db39063 >2013-12-12 13:58:22,506 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7db39063 >2013-12-12 13:58:22,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 172dfc4e >2013-12-12 13:58:22,685 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-75) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d095aac5}, log id: 172dfc4e >2013-12-12 13:58:27,695 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5fa71151 >2013-12-12 13:58:28,035 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5fa71151 >2013-12-12 13:58:28,037 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 79d15cbb >2013-12-12 13:58:28,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-86) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1baf5733}, log id: 79d15cbb >2013-12-12 13:58:33,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3514316d >2013-12-12 13:58:33,559 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3514316d >2013-12-12 13:58:33,560 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 54ecaf03 >2013-12-12 13:58:33,738 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2722577d}, log id: 54ecaf03 >2013-12-12 13:58:38,747 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 620be419 >2013-12-12 13:58:39,089 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 620be419 >2013-12-12 13:58:39,090 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4c1bd7d5 >2013-12-12 13:58:39,267 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@89ed9b8b}, log id: 4c1bd7d5 >2013-12-12 13:58:44,277 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 71da99b8 >2013-12-12 13:58:44,615 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 71da99b8 >2013-12-12 13:58:44,617 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 15723b2a >2013-12-12 13:58:44,797 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7587ac8c}, log id: 15723b2a >2013-12-12 13:58:49,806 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 615aa9c8 >2013-12-12 13:58:50,137 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 615aa9c8 >2013-12-12 13:58:50,140 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4a0b3a6f >2013-12-12 13:58:50,318 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a13301b4}, log id: 4a0b3a6f >2013-12-12 13:58:55,330 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3236e0cb >2013-12-12 13:58:55,674 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3236e0cb >2013-12-12 13:58:55,676 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 400260c >2013-12-12 13:58:55,856 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c43437e9}, log id: 400260c >2013-12-12 13:59:00,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6c9336df >2013-12-12 13:59:01,204 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6c9336df >2013-12-12 13:59:01,205 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2a7afa5b >2013-12-12 13:59:01,381 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cdf072d7}, log id: 2a7afa5b >2013-12-12 13:59:03,220 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-33) No up server in cluster >2013-12-12 13:59:03,220 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-33) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 13:59:03,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 8db07c4 >2013-12-12 13:59:03,407 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GlusterTasksListVDSCommand, return: [], log id: 8db07c4 >2013-12-12 13:59:06,411 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: bcf5f20 >2013-12-12 13:59:06,750 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: bcf5f20 >2013-12-12 13:59:06,752 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 26660927 >2013-12-12 13:59:06,930 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@65d5c1b5}, log id: 26660927 >2013-12-12 13:59:11,943 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 15615024 >2013-12-12 13:59:12,288 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 15615024 >2013-12-12 13:59:12,290 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7fb60535 >2013-12-12 13:59:12,469 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@914d1f86}, log id: 7fb60535 >2013-12-12 13:59:17,480 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1d65580a >2013-12-12 13:59:17,819 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1d65580a >2013-12-12 13:59:17,821 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 55dcd >2013-12-12 13:59:18,000 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a8e74366}, log id: 55dcd >2013-12-12 13:59:23,015 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 72202b96 >2013-12-12 13:59:23,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 72202b96 >2013-12-12 13:59:23,351 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7da36754 >2013-12-12 13:59:23,532 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d660a600}, log id: 7da36754 >2013-12-12 13:59:28,543 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 216278ff >2013-12-12 13:59:28,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 216278ff >2013-12-12 13:59:28,880 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4012b47a >2013-12-12 13:59:29,059 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e3489bca}, log id: 4012b47a >2013-12-12 13:59:34,070 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 6265d28e >2013-12-12 13:59:34,410 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 6265d28e >2013-12-12 13:59:34,412 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 70db4c3a >2013-12-12 13:59:34,592 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ded46ae2}, log id: 70db4c3a >2013-12-12 13:59:39,602 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: fe1523d >2013-12-12 13:59:39,932 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: fe1523d >2013-12-12 13:59:39,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 64fa8407 >2013-12-12 13:59:40,107 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1f1001d}, log id: 64fa8407 >2013-12-12 13:59:45,118 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 73d1ab38 >2013-12-12 13:59:45,453 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 73d1ab38 >2013-12-12 13:59:45,454 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 685c19ef >2013-12-12 13:59:45,634 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6a71e086}, log id: 685c19ef >2013-12-12 13:59:50,648 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-97) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4bb732e1 >2013-12-12 13:59:50,983 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-97) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 4bb732e1 >2013-12-12 13:59:50,984 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-97) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 18cf283f >2013-12-12 13:59:51,162 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-97) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1ddf6477}, log id: 18cf283f >2013-12-12 13:59:56,172 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 56fadbf8 >2013-12-12 13:59:56,505 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 56fadbf8 >2013-12-12 13:59:56,506 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1910b66f >2013-12-12 13:59:56,684 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@15899fa6}, log id: 1910b66f >2013-12-12 14:00:01,696 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 46dce416 >2013-12-12 14:00:02,029 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 46dce416 >2013-12-12 14:00:02,031 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5e551c10 >2013-12-12 14:00:02,205 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b0a6c0f2}, log id: 5e551c10 >2013-12-12 14:00:03,409 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-18) No up server in cluster >2013-12-12 14:00:03,409 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-18) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:00:03,412 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-18) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7b1808e9 >2013-12-12 14:00:03,587 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, GlusterTasksListVDSCommand, return: [], log id: 7b1808e9 >2013-12-12 14:00:07,215 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1464f4ca >2013-12-12 14:00:07,549 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1464f4ca >2013-12-12 14:00:07,551 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: a8c65f >2013-12-12 14:00:07,727 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-23) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6c164e22}, log id: a8c65f >2013-12-12 14:00:12,736 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 6eebe288 >2013-12-12 14:00:13,072 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 6eebe288 >2013-12-12 14:00:13,073 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7272cfe4 >2013-12-12 14:00:13,249 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-29) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fe083018}, log id: 7272cfe4 >2013-12-12 14:00:18,260 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4db0f1c6 >2013-12-12 14:00:18,595 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 4db0f1c6 >2013-12-12 14:00:18,597 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7f412a51 >2013-12-12 14:00:18,774 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@660457cb}, log id: 7f412a51 >2013-12-12 14:00:23,787 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 30f6faab >2013-12-12 14:00:24,121 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 30f6faab >2013-12-12 14:00:24,123 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3e8b8cfb >2013-12-12 14:00:24,300 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@70cc6576}, log id: 3e8b8cfb >2013-12-12 14:00:29,310 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: d48d8d >2013-12-12 14:00:29,644 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: d48d8d >2013-12-12 14:00:29,646 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4c3f735e >2013-12-12 14:00:29,823 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@709b6413}, log id: 4c3f735e >2013-12-12 14:00:34,832 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 5e97fff7 >2013-12-12 14:00:35,163 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 5e97fff7 >2013-12-12 14:00:35,165 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 775e5f30 >2013-12-12 14:00:35,345 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bb67da29}, log id: 775e5f30 >2013-12-12 14:00:40,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1f80d472 >2013-12-12 14:00:40,685 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1f80d472 >2013-12-12 14:00:40,687 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 20b2731b >2013-12-12 14:00:40,864 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@24f90001}, log id: 20b2731b >2013-12-12 14:00:45,878 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 53d974fa >2013-12-12 14:00:46,219 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 53d974fa >2013-12-12 14:00:46,221 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 17d704f0 >2013-12-12 14:00:46,395 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@36a79e41}, log id: 17d704f0 >2013-12-12 14:00:51,406 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2b3ddb89 >2013-12-12 14:00:51,743 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 2b3ddb89 >2013-12-12 14:00:51,745 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 74381703 >2013-12-12 14:00:51,923 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2af537a4}, log id: 74381703 >2013-12-12 14:00:56,933 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2f7c1088 >2013-12-12 14:00:57,264 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2f7c1088 >2013-12-12 14:00:57,266 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 341548d4 >2013-12-12 14:00:57,443 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@250ea03c}, log id: 341548d4 >2013-12-12 14:00:59,320 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 304eaccc >2013-12-12 14:00:59,508 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@7a4cd5d8, log id: 304eaccc >2013-12-12 14:01:02,453 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1d5a5735 >2013-12-12 14:01:02,982 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1d5a5735 >2013-12-12 14:01:02,984 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5feb041d >2013-12-12 14:01:03,257 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6c68e4a8}, log id: 5feb041d >2013-12-12 14:01:03,589 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-4) No up server in cluster >2013-12-12 14:01:03,589 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-4) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:01:03,592 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GlusterTasksListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 395f7cd >2013-12-12 14:01:03,940 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GlusterTasksListVDSCommand, return: [], log id: 395f7cd >2013-12-12 14:01:08,268 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 5e20f1ce >2013-12-12 14:01:08,743 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 5e20f1ce >2013-12-12 14:01:08,745 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 328f5c90 >2013-12-12 14:01:08,997 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@eb85715d}, log id: 328f5c90 >2013-12-12 14:01:14,006 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1243805b >2013-12-12 14:01:14,486 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1243805b >2013-12-12 14:01:14,488 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 24ad2fcd >2013-12-12 14:01:14,736 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@92ee828d}, log id: 24ad2fcd >2013-12-12 14:01:19,745 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 12e2e0d0 >2013-12-12 14:01:20,225 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 12e2e0d0 >2013-12-12 14:01:20,226 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: a7fee4 >2013-12-12 14:01:20,479 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bf00607b}, log id: a7fee4 >2013-12-12 14:01:22,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Used CPU of host node2 [99%] exceeded defined threshold [95%]. >2013-12-12 14:01:22,739 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Used CPU of host node1 [99%] exceeded defined threshold [95%]. >2013-12-12 14:01:25,489 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7efb472f >2013-12-12 14:01:25,830 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7efb472f >2013-12-12 14:01:25,832 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 3a2fb4bd >2013-12-12 14:01:26,007 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2489786c}, log id: 3a2fb4bd >2013-12-12 14:01:31,017 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7a246789 >2013-12-12 14:01:31,348 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7a246789 >2013-12-12 14:01:31,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 437fa336 >2013-12-12 14:01:31,526 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6dbd3e86}, log id: 437fa336 >2013-12-12 14:01:36,537 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1fb2d1ae >2013-12-12 14:01:36,874 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1fb2d1ae >2013-12-12 14:01:36,875 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2728a673 >2013-12-12 14:01:37,056 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-43) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ce216ab2}, log id: 2728a673 >2013-12-12 14:01:42,066 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7db2747c >2013-12-12 14:01:42,399 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 7db2747c >2013-12-12 14:01:42,401 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 516bf276 >2013-12-12 14:01:42,581 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-55) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c9b33050}, log id: 516bf276 >2013-12-12 14:01:47,592 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7d5d80d7 >2013-12-12 14:01:47,932 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 7d5d80d7 >2013-12-12 14:01:47,934 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2812c494 >2013-12-12 14:01:48,114 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6a90380a}, log id: 2812c494 >2013-12-12 14:01:53,153 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 652c54d6 >2013-12-12 14:01:53,503 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 652c54d6 >2013-12-12 14:01:53,505 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 7ed8fe50 >2013-12-12 14:01:53,684 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ca533045}, log id: 7ed8fe50 >2013-12-12 14:01:58,695 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1d024771 >2013-12-12 14:01:59,027 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1d024771 >2013-12-12 14:01:59,029 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 65f7eed >2013-12-12 14:01:59,212 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@79944f87}, log id: 65f7eed >2013-12-12 14:02:03,943 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-83) No up server in cluster >2013-12-12 14:02:03,943 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-83) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:02:03,946 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-83) START, GlusterTasksListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3eedb46b >2013-12-12 14:02:04,124 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, GlusterTasksListVDSCommand, return: [], log id: 3eedb46b >2013-12-12 14:02:04,222 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2446a8bf >2013-12-12 14:02:04,563 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 2446a8bf >2013-12-12 14:02:04,565 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 927ea15 >2013-12-12 14:02:04,743 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d5683fef}, log id: 927ea15 >2013-12-12 14:02:09,766 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 125557b0 >2013-12-12 14:02:10,100 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 125557b0 >2013-12-12 14:02:10,101 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 66ecf56b >2013-12-12 14:02:10,278 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1f7afbd3}, log id: 66ecf56b >2013-12-12 14:02:15,287 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3cb257d7 >2013-12-12 14:02:15,619 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 3cb257d7 >2013-12-12 14:02:15,620 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 2d97eb13 >2013-12-12 14:02:15,799 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b97c27a4}, log id: 2d97eb13 >2013-12-12 14:02:20,809 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 466ace05 >2013-12-12 14:02:21,145 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 466ace05 >2013-12-12 14:02:21,147 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 695d923f >2013-12-12 14:02:21,322 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8f209a24}, log id: 695d923f >2013-12-12 14:02:26,332 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 1ec6740e >2013-12-12 14:02:26,677 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 1ec6740e >2013-12-12 14:02:26,678 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 48a0de1a >2013-12-12 14:02:26,855 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a554bf9f}, log id: 48a0de1a >2013-12-12 14:02:31,866 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 788b0bcd >2013-12-12 14:02:32,202 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 788b0bcd >2013-12-12 14:02:32,203 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 39017bd6 >2013-12-12 14:02:32,387 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@15512d12}, log id: 39017bd6 >2013-12-12 14:02:37,398 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1e944a3d >2013-12-12 14:02:37,737 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: 1e944a3d >2013-12-12 14:02:37,739 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: a6c5464 >2013-12-12 14:02:37,919 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5de46c57}, log id: a6c5464 >2013-12-12 14:02:42,928 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: b06efa8 >2013-12-12 14:02:43,269 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterServersListVDSCommand, return: [10.14.16.132:CONNECTED, latest-b:CONNECTED], log id: b06efa8 >2013-12-12 14:02:43,271 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 16c34998 >2013-12-12 14:02:43,454 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@37115a93}, log id: 16c34998 >2013-12-12 14:02:45,260 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2013-12-12 14:02:45,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2013-12-12 14:02:45,368 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-12) [5f79bf03] Lock Acquired to object EngineLock [exclusiveLocks= key: 6504a995-e161-4543-8ae6-ccb36f29383e value: GLUSTER >, sharedLocks= ] >2013-12-12 14:02:45,387 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-12) [5f79bf03] Running command: StartRebalanceGlusterVolumeCommand internal: false. Entities affected : ID: 6504a995-e161-4543-8ae6-ccb36f29383e Type: GlusterVolume >2013-12-12 14:02:45,397 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-12) [5f79bf03] START, StartRebalanceGlusterVolumeVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1c03d825 >2013-12-12 14:02:48,463 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 676c9771 >2013-12-12 14:02:48,803 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterServersListVDSCommand, return: [10.14.16.127:CONNECTED, 10.14.16.132:CONNECTED], log id: 676c9771 >2013-12-12 14:02:48,804 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 527ec83c >2013-12-12 14:02:48,983 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GlusterVolumesListVDSCommand, return: {6504a995-e161-4543-8ae6-ccb36f29383e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fdd663d}, log id: 527ec83c >2013-12-12 14:02:50,777 INFO [org.ovirt.engine.core.vdsbroker.gluster.StartRebalanceGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-12) [5f79bf03] FINISH, StartRebalanceGlusterVolumeVDSCommand, return: org.ovirt.engine.core.common.asynctasks.gluster.GlusterAsyncTask@67e12cdf, log id: 1c03d825 >2013-12-12 14:02:50,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [5f79bf03] Correlation ID: 5f79bf03, Job ID: b34bc99f-753b-4771-a1ad-7ddb9367e1f7, Call Stack: null, Custom Event ID: -1, Message: Could not start Gluster Volume StartMigrationDuringRebalanceTest rebalance. >2013-12-12 14:02:50,836 INFO [org.ovirt.engine.core.bll.gluster.StartRebalanceGlusterVolumeCommand] (ajp-/127.0.0.1:8702-12) [5f79bf03] Lock freed to object EngineLock [exclusiveLocks= key: 6504a995-e161-4543-8ae6-ccb36f29383e value: GLUSTER >, sharedLocks= ] >2013-12-12 14:02:50,840 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-12) Operation Failed: [command execution failed >error: connection failed. please check if gluster daemon is operational. >return code: 1] >2013-12-12 14:02:50,846 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [33de7eaf] Running command: LogoutUserCommand internal: false. >2013-12-12 14:02:50,862 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [33de7eaf] Correlation ID: 33de7eaf, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2013-12-12 14:02:54,007 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 629c2b20 >2013-12-12 14:02:55,939 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GlusterServersListVDSCommand, log id: 629c2b20 >2013-12-12 14:02:55,940 INFO [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-49) Gluster servers list failed in server node2 moving it to NonOperational >2013-12-12 14:02:56,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster peer list from server node2 on Cluster myCluster. >2013-12-12 14:02:56,279 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [7a10f52f] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:02:56,280 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [7a10f52f] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 270a30bb >2013-12-12 14:02:56,287 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [7a10f52f] FINISH, SetVdsStatusVDSCommand, log id: 270a30bb >2013-12-12 14:02:56,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [7a10f52f] Correlation ID: 7a10f52f, Job ID: 8500dae8-4501-49fc-a601-a24f03984ace, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node2. >2013-12-12 14:02:56,330 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) [7a10f52f] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 722c769d >2013-12-12 14:02:59,344 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-49) [7a10f52f] FINISH, GlusterServersListVDSCommand, log id: 722c769d >2013-12-12 14:02:59,344 INFO [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-49) [7a10f52f] Gluster servers list failed in server node1 moving it to NonOperational >2013-12-12 14:02:59,354 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [7a10f52f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster peer list from server node1 on Cluster myCluster. >2013-12-12 14:02:59,395 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [5b5b167c] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:02:59,397 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [5b5b167c] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 3a5c9954 >2013-12-12 14:02:59,404 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [5b5b167c] FINISH, SetVdsStatusVDSCommand, log id: 3a5c9954 >2013-12-12 14:02:59,421 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [5b5b167c] Correlation ID: 5b5b167c, Job ID: 112f9432-06f0-4f9b-98b6-03fc974a4f12, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node1. >2013-12-12 14:02:59,429 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-49) [5b5b167c] gluster peer status command failed on all servers of the cluster myCluster.Cant refresh its data at this point. >2013-12-12 14:02:59,430 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) [5b5b167c] START, GlusterVolumesListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3ec30787 >2013-12-12 14:03:02,444 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) [5b5b167c] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 3ec30787 >2013-12-12 14:03:02,454 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [5b5b167c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster volume list from server node2. >2013-12-12 14:03:02,487 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:03:02,488 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 18981cc4 >2013-12-12 14:03:02,504 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] FINISH, SetVdsStatusVDSCommand, log id: 18981cc4 >2013-12-12 14:03:02,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] Correlation ID: 2b9ea61c, Job ID: 2ad188a0-7ed7-462a-a1f5-8c0f9cc22c67, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node2. >2013-12-12 14:03:02,530 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] START, GlusterVolumesListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 2bc7777e >2013-12-12 14:03:04,127 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:03:04,127 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:03:04,128 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:03:04,128 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:03:05,545 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 2bc7777e >2013-12-12 14:03:05,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [2b9ea61c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster volume list from server node1. >2013-12-12 14:03:05,596 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [7d1e4632] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:03:05,597 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [7d1e4632] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 5c1be914 >2013-12-12 14:03:05,604 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [7d1e4632] FINISH, SetVdsStatusVDSCommand, log id: 5c1be914 >2013-12-12 14:03:05,621 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [7d1e4632] Correlation ID: 7d1e4632, Job ID: 9a35dfc8-5ee0-4116-9050-d607a34c52f7, Call Stack: null, Custom Event ID: -1, Message: Gluster command [gluster peer status] failed on server node1. >2013-12-12 14:03:05,629 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-49) [7d1e4632] gluster volume info command failed on all servers of the cluster myCluster.Cant refresh its data at this point. >2013-12-12 14:03:06,293 WARN [org.ovirt.engine.core.branding.BrandingTheme] (ajp-/127.0.0.1:8702-4) Theme /80-rhs-1.brand has no property defined for key resources >2013-12-12 14:04:04,133 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-48) No up server in cluster >2013-12-12 14:04:04,133 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-48) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:04:04,135 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-48) No up server in cluster >2013-12-12 14:04:04,135 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-48) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:05:00,008 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-22) Autorecovering 2 hosts >2013-12-12 14:05:00,008 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-22) Autorecovering hosts id: a77a7c9d-6a77-4f57-8db1-8162e4d58fff, name : node1 >2013-12-12 14:05:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] Lock Acquired to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:05:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] Running command: ActivateVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:05:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] Before acquiring lock in order to prevent monitoring for host node1 from data-center Default >2013-12-12 14:05:00,013 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] Lock acquired, from now a monitoring of host will be skipped for host node1 from data-center Default >2013-12-12 14:05:00,021 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Unassigned, nonOperationalReason=NONE), log id: 1a459728 >2013-12-12 14:05:00,068 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] FINISH, SetVdsStatusVDSCommand, log id: 1a459728 >2013-12-12 14:05:00,070 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] START, ActivateVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 10e18ed3 >2013-12-12 14:05:00,106 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 225568d9 >2013-12-12 14:05:00,114 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] FINISH, GetHardwareInfoVDSCommand, log id: 225568d9 >2013-12-12 14:05:00,128 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 31c7da97 >2013-12-12 14:05:03,146 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) [2ca1626f] FINISH, GlusterServersListVDSCommand, log id: 31c7da97 >2013-12-12 14:05:03,194 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-22) [235f1f31] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:05:03,196 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [235f1f31] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 54b0d930 >2013-12-12 14:05:03,210 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [235f1f31] FINISH, SetVdsStatusVDSCommand, log id: 54b0d930 >2013-12-12 14:05:03,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [235f1f31] Correlation ID: 235f1f31, Job ID: b1da7581-702d-4d6c-99a0-3c2a51bb9b51, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node1. >2013-12-12 14:05:03,245 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-22) [541b3245] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:05:03,245 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-22) [541b3245] FINISH, ActivateVdsVDSCommand, return: Host[node1], log id: 10e18ed3 >2013-12-12 14:05:03,277 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [541b3245] Activate finished. Lock released. Monitoring can run now for host node1 from data-center Default >2013-12-12 14:05:03,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [541b3245] Correlation ID: 2ca1626f, Call Stack: null, Custom Event ID: -1, Message: Host node1 was autorecovered. >2013-12-12 14:05:03,286 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [541b3245] Lock freed to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:05:03,286 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-22) [541b3245] Autorecovering hosts id: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, name : node2 >2013-12-12 14:05:03,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] Lock Acquired to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:05:03,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:05:03,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] Before acquiring lock in order to prevent monitoring for host node2 from data-center Default >2013-12-12 14:05:03,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] Lock acquired, from now a monitoring of host will be skipped for host node2 from data-center Default >2013-12-12 14:05:03,290 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Unassigned, nonOperationalReason=NONE), log id: 79b22698 >2013-12-12 14:05:03,302 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] FINISH, SetVdsStatusVDSCommand, log id: 79b22698 >2013-12-12 14:05:03,310 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] START, ActivateVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 364f8bf2 >2013-12-12 14:05:03,367 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: 67f2d1ad >2013-12-12 14:05:03,374 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] FINISH, GetHardwareInfoVDSCommand, log id: 67f2d1ad >2013-12-12 14:05:03,387 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 34df4583 >2013-12-12 14:05:04,137 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-30) [64e77997] No up server in cluster >2013-12-12 14:05:04,137 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-30) [64e77997] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:05:04,138 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-30) [64e77997] No up server in cluster >2013-12-12 14:05:04,138 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-30) [64e77997] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:05:06,401 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-22) [7d343e0d] FINISH, GlusterServersListVDSCommand, log id: 34df4583 >2013-12-12 14:05:06,443 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-22) [6665fec4] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:05:06,445 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [6665fec4] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 1882fff2 >2013-12-12 14:05:06,452 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) [6665fec4] FINISH, SetVdsStatusVDSCommand, log id: 1882fff2 >2013-12-12 14:05:06,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [6665fec4] Correlation ID: 6665fec4, Job ID: 677715ed-30a8-4230-a7e1-14e45fc104ed, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node2. >2013-12-12 14:05:06,486 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-22) [182a4f4b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:05:06,487 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-22) [182a4f4b] FINISH, ActivateVdsVDSCommand, return: Host[node2], log id: 364f8bf2 >2013-12-12 14:05:06,510 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [182a4f4b] Activate finished. Lock released. Monitoring can run now for host node2 from data-center Default >2013-12-12 14:05:06,518 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [182a4f4b] Correlation ID: 7d343e0d, Call Stack: null, Custom Event ID: -1, Message: Host node2 was autorecovered. >2013-12-12 14:05:06,519 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-22) [182a4f4b] Lock freed to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:06:04,141 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-13) No up server in cluster >2013-12-12 14:06:04,141 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-13) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:06:04,142 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-13) No up server in cluster >2013-12-12 14:06:04,142 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-13) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:07:04,144 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster >2013-12-12 14:07:04,144 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-95) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:07:04,145 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster >2013-12-12 14:07:04,145 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-95) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:08:04,147 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-78) No up server in cluster >2013-12-12 14:08:04,147 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-78) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:08:04,148 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-78) No up server in cluster >2013-12-12 14:08:04,148 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-78) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:09:04,151 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:09:04,151 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:09:04,152 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:09:04,152 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:10:00,006 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-37) Autorecovering 2 hosts >2013-12-12 14:10:00,007 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-37) Autorecovering hosts id: a77a7c9d-6a77-4f57-8db1-8162e4d58fff, name : node1 >2013-12-12 14:10:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [1363351] Lock Acquired to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:10:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [1363351] Running command: ActivateVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:10:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [1363351] Before acquiring lock in order to prevent monitoring for host node1 from data-center Default >2013-12-12 14:10:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [1363351] Lock acquired, from now a monitoring of host will be skipped for host node1 from data-center Default >2013-12-12 14:10:00,012 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Unassigned, nonOperationalReason=NONE), log id: 1483d622 >2013-12-12 14:10:00,021 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] FINISH, SetVdsStatusVDSCommand, log id: 1483d622 >2013-12-12 14:10:00,022 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] START, ActivateVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 49f4cae8 >2013-12-12 14:10:00,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 2c9fbfa8 >2013-12-12 14:10:00,064 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] FINISH, GetHardwareInfoVDSCommand, log id: 2c9fbfa8 >2013-12-12 14:10:00,081 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4fe594ed >2013-12-12 14:10:03,095 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) [1363351] FINISH, GlusterServersListVDSCommand, log id: 4fe594ed >2013-12-12 14:10:03,137 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-37) [b5c9f2d] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:10:03,139 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [b5c9f2d] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 37f0a95e >2013-12-12 14:10:03,146 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [b5c9f2d] FINISH, SetVdsStatusVDSCommand, log id: 37f0a95e >2013-12-12 14:10:03,163 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-37) [b5c9f2d] Correlation ID: b5c9f2d, Job ID: 84197c3f-8a0b-4790-8370-a9824ff8150d, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node1. >2013-12-12 14:10:03,186 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-37) [55a7ebef] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:10:03,187 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-37) [55a7ebef] FINISH, ActivateVdsVDSCommand, return: Host[node1], log id: 49f4cae8 >2013-12-12 14:10:03,213 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [55a7ebef] Activate finished. Lock released. Monitoring can run now for host node1 from data-center Default >2013-12-12 14:10:03,214 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [55a7ebef] Lock freed to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:10:03,214 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-37) [55a7ebef] Autorecovering hosts id: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, name : node2 >2013-12-12 14:10:03,216 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] Lock Acquired to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:10:03,216 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:10:03,216 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] Before acquiring lock in order to prevent monitoring for host node2 from data-center Default >2013-12-12 14:10:03,216 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] Lock acquired, from now a monitoring of host will be skipped for host node2 from data-center Default >2013-12-12 14:10:03,222 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Unassigned, nonOperationalReason=NONE), log id: 360d1556 >2013-12-12 14:10:03,229 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] FINISH, SetVdsStatusVDSCommand, log id: 360d1556 >2013-12-12 14:10:03,230 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] START, ActivateVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 4de22f48 >2013-12-12 14:10:03,267 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: 6d2893 >2013-12-12 14:10:03,274 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] FINISH, GetHardwareInfoVDSCommand, log id: 6d2893 >2013-12-12 14:10:03,291 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: cd7c21f >2013-12-12 14:10:04,154 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-54) No up server in cluster >2013-12-12 14:10:04,154 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-54) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:10:04,155 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-54) No up server in cluster >2013-12-12 14:10:04,155 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-54) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:10:06,305 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-37) [697068ad] FINISH, GlusterServersListVDSCommand, log id: cd7c21f >2013-12-12 14:10:06,421 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-37) [26392d0a] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:10:06,423 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [26392d0a] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 787ca33e >2013-12-12 14:10:06,429 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) [26392d0a] FINISH, SetVdsStatusVDSCommand, log id: 787ca33e >2013-12-12 14:10:06,530 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-37) [26392d0a] Correlation ID: 26392d0a, Job ID: b22abefc-33cb-45e1-8019-6b92e206b934, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node2. >2013-12-12 14:10:06,555 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-37) [72ccba09] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:10:06,555 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-37) [72ccba09] FINISH, ActivateVdsVDSCommand, return: Host[node2], log id: 4de22f48 >2013-12-12 14:10:06,571 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [72ccba09] Activate finished. Lock released. Monitoring can run now for host node2 from data-center Default >2013-12-12 14:10:06,572 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-37) [72ccba09] Lock freed to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:11:04,157 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-34) No up server in cluster >2013-12-12 14:11:04,158 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-34) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:11:04,159 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-34) No up server in cluster >2013-12-12 14:11:04,159 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-34) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:12:04,161 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 14:12:04,161 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:12:04,162 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 14:12:04,162 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:13:04,164 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-96) No up server in cluster >2013-12-12 14:13:04,164 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-96) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:13:04,165 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-96) No up server in cluster >2013-12-12 14:13:04,165 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-96) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:14:04,167 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-85) No up server in cluster >2013-12-12 14:14:04,168 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-85) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:14:04,169 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-85) No up server in cluster >2013-12-12 14:14:04,169 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-85) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:15:00,007 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-62) Autorecovering 2 hosts >2013-12-12 14:15:00,008 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-62) Autorecovering hosts id: a77a7c9d-6a77-4f57-8db1-8162e4d58fff, name : node1 >2013-12-12 14:15:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] Lock Acquired to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:15:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] Running command: ActivateVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:15:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] Before acquiring lock in order to prevent monitoring for host node1 from data-center Default >2013-12-12 14:15:00,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] Lock acquired, from now a monitoring of host will be skipped for host node1 from data-center Default >2013-12-12 14:15:00,015 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Unassigned, nonOperationalReason=NONE), log id: 5d139333 >2013-12-12 14:15:00,033 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] FINISH, SetVdsStatusVDSCommand, log id: 5d139333 >2013-12-12 14:15:00,034 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] START, ActivateVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 25db8966 >2013-12-12 14:15:00,070 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 3736de06 >2013-12-12 14:15:00,079 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] FINISH, GetHardwareInfoVDSCommand, log id: 3736de06 >2013-12-12 14:15:00,095 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 1c56cde2 >2013-12-12 14:15:03,109 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) [3d51e595] FINISH, GlusterServersListVDSCommand, log id: 1c56cde2 >2013-12-12 14:15:03,166 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-62) [117ed4e8] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:15:03,167 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [117ed4e8] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: a410cef >2013-12-12 14:15:03,174 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [117ed4e8] FINISH, SetVdsStatusVDSCommand, log id: a410cef >2013-12-12 14:15:03,200 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) [117ed4e8] Correlation ID: 117ed4e8, Job ID: 533ff0e5-89e3-4aa7-8377-dac8e25c9e41, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node1. >2013-12-12 14:15:03,218 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-62) [2ebc5d95] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:15:03,218 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-62) [2ebc5d95] FINISH, ActivateVdsVDSCommand, return: Host[node1], log id: 25db8966 >2013-12-12 14:15:03,237 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [2ebc5d95] Activate finished. Lock released. Monitoring can run now for host node1 from data-center Default >2013-12-12 14:15:03,238 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [2ebc5d95] Lock freed to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:15:03,238 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-62) [2ebc5d95] Autorecovering hosts id: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, name : node2 >2013-12-12 14:15:03,239 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] Lock Acquired to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:15:03,239 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:15:03,239 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] Before acquiring lock in order to prevent monitoring for host node2 from data-center Default >2013-12-12 14:15:03,239 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] Lock acquired, from now a monitoring of host will be skipped for host node2 from data-center Default >2013-12-12 14:15:03,242 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Unassigned, nonOperationalReason=NONE), log id: 60d8d04 >2013-12-12 14:15:03,250 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] FINISH, SetVdsStatusVDSCommand, log id: 60d8d04 >2013-12-12 14:15:03,258 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] START, ActivateVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 407fd4de >2013-12-12 14:15:03,301 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: 33969cd8 >2013-12-12 14:15:03,308 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] FINISH, GetHardwareInfoVDSCommand, log id: 33969cd8 >2013-12-12 14:15:03,326 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 3b49e8e3 >2013-12-12 14:15:04,171 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:15:04,172 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:15:04,173 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-66) No up server in cluster >2013-12-12 14:15:04,173 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-66) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:15:06,341 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-62) [50fc60c] FINISH, GlusterServersListVDSCommand, log id: 3b49e8e3 >2013-12-12 14:15:06,400 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-62) [45c998c4] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:15:06,401 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [45c998c4] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 3e48ee1 >2013-12-12 14:15:06,408 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) [45c998c4] FINISH, SetVdsStatusVDSCommand, log id: 3e48ee1 >2013-12-12 14:15:06,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) [45c998c4] Correlation ID: 45c998c4, Job ID: b678a514-c3c6-465e-b4bb-54a0ef5e3268, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node2. >2013-12-12 14:15:06,451 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-62) [529dfa68] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:15:06,452 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-62) [529dfa68] FINISH, ActivateVdsVDSCommand, return: Host[node2], log id: 407fd4de >2013-12-12 14:15:06,483 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [529dfa68] Activate finished. Lock released. Monitoring can run now for host node2 from data-center Default >2013-12-12 14:15:06,484 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-62) [529dfa68] Lock freed to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:16:04,175 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-37) [72ccba09] No up server in cluster >2013-12-12 14:16:04,176 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-37) [72ccba09] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:16:04,178 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-37) [72ccba09] No up server in cluster >2013-12-12 14:16:04,178 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-37) [72ccba09] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:17:04,181 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-31) No up server in cluster >2013-12-12 14:17:04,181 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-31) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:17:04,182 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-31) No up server in cluster >2013-12-12 14:17:04,182 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-31) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:18:04,184 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 14:18:04,184 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:18:04,185 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-15) No up server in cluster >2013-12-12 14:18:04,186 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-15) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:19:04,190 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster >2013-12-12 14:19:04,190 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-95) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:19:04,191 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster >2013-12-12 14:19:04,191 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-95) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:20:00,006 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-75) Autorecovering 2 hosts >2013-12-12 14:20:00,007 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-75) Autorecovering hosts id: a77a7c9d-6a77-4f57-8db1-8162e4d58fff, name : node1 >2013-12-12 14:20:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] Lock Acquired to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:20:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] Running command: ActivateVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:20:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] Before acquiring lock in order to prevent monitoring for host node1 from data-center Default >2013-12-12 14:20:00,008 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] Lock acquired, from now a monitoring of host will be skipped for host node1 from data-center Default >2013-12-12 14:20:00,012 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=Unassigned, nonOperationalReason=NONE), log id: 499c40e >2013-12-12 14:20:00,087 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] FINISH, SetVdsStatusVDSCommand, log id: 499c40e >2013-12-12 14:20:00,088 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] START, ActivateVdsVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4a094c0 >2013-12-12 14:20:00,124 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] START, GetHardwareInfoVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, vds=Host[node1]), log id: 233c511f >2013-12-12 14:20:00,131 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] FINISH, GetHardwareInfoVDSCommand, log id: 233c511f >2013-12-12 14:20:00,145 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] START, GlusterServersListVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff), log id: 4e0a6422 >2013-12-12 14:20:03,160 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) [2bec47b8] FINISH, GlusterServersListVDSCommand, log id: 4e0a6422 >2013-12-12 14:20:03,210 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-75) [6397685f] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:20:03,211 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [6397685f] START, SetVdsStatusVDSCommand(HostName = node1, HostId = a77a7c9d-6a77-4f57-8db1-8162e4d58fff, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 242070a2 >2013-12-12 14:20:03,219 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [6397685f] FINISH, SetVdsStatusVDSCommand, log id: 242070a2 >2013-12-12 14:20:03,236 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) [6397685f] Correlation ID: 6397685f, Job ID: 9018801e-7d32-4d45-b1da-307f86d42f99, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node1. >2013-12-12 14:20:03,253 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-75) [35c50850] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: a77a7c9d-6a77-4f57-8db1-8162e4d58fff Type: VDS >2013-12-12 14:20:03,254 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-75) [35c50850] FINISH, ActivateVdsVDSCommand, return: Host[node1], log id: 4a094c0 >2013-12-12 14:20:03,277 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [35c50850] Activate finished. Lock released. Monitoring can run now for host node1 from data-center Default >2013-12-12 14:20:03,278 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [35c50850] Lock freed to object EngineLock [exclusiveLocks= key: a77a7c9d-6a77-4f57-8db1-8162e4d58fff value: VDS >, sharedLocks= ] >2013-12-12 14:20:03,278 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-75) [35c50850] Autorecovering hosts id: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, name : node2 >2013-12-12 14:20:03,280 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] Lock Acquired to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:20:03,280 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:20:03,280 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] Before acquiring lock in order to prevent monitoring for host node2 from data-center Default >2013-12-12 14:20:03,280 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] Lock acquired, from now a monitoring of host will be skipped for host node2 from data-center Default >2013-12-12 14:20:03,282 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=Unassigned, nonOperationalReason=NONE), log id: 7746759e >2013-12-12 14:20:03,294 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] FINISH, SetVdsStatusVDSCommand, log id: 7746759e >2013-12-12 14:20:03,295 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] START, ActivateVdsVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 34bc3028 >2013-12-12 14:20:03,331 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] START, GetHardwareInfoVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, vds=Host[node2]), log id: 6bfdb165 >2013-12-12 14:20:03,340 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] FINISH, GetHardwareInfoVDSCommand, log id: 6bfdb165 >2013-12-12 14:20:03,355 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] START, GlusterServersListVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20), log id: 7131971d >2013-12-12 14:20:04,194 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-78) No up server in cluster >2013-12-12 14:20:04,194 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-78) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:20:04,197 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-78) No up server in cluster >2013-12-12 14:20:04,197 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-78) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:20:06,369 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-75) [1d0cafc3] FINISH, GlusterServersListVDSCommand, log id: 7131971d >2013-12-12 14:20:06,535 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-75) [797bfd89] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:20:06,537 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [797bfd89] START, SetVdsStatusVDSCommand(HostName = node2, HostId = 60cc0e1d-bcb1-429b-9a15-aab41a70dc20, status=NonOperational, nonOperationalReason=GLUSTER_COMMAND_FAILED), log id: 195ed51f >2013-12-12 14:20:06,544 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) [797bfd89] FINISH, SetVdsStatusVDSCommand, log id: 195ed51f >2013-12-12 14:20:06,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) [797bfd89] Correlation ID: 797bfd89, Job ID: e7cac8fb-76c8-4a4f-a35d-b6c6e3b3cb58, Call Stack: null, Custom Event ID: -1, Message: Gluster command [<UNKNOWN>] failed on server node2. >2013-12-12 14:20:06,594 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-75) [4a1ab0e6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 Type: VDS >2013-12-12 14:20:06,595 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (DefaultQuartzScheduler_Worker-75) [4a1ab0e6] FINISH, ActivateVdsVDSCommand, return: Host[node2], log id: 34bc3028 >2013-12-12 14:20:06,619 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [4a1ab0e6] Activate finished. Lock released. Monitoring can run now for host node2 from data-center Default >2013-12-12 14:20:06,620 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-75) [4a1ab0e6] Lock freed to object EngineLock [exclusiveLocks= key: 60cc0e1d-bcb1-429b-9a15-aab41a70dc20 value: VDS >, sharedLocks= ] >2013-12-12 14:21:04,201 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-60) [44ea9dad] No up server in cluster >2013-12-12 14:21:04,201 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-60) [44ea9dad] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:21:04,203 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-60) [44ea9dad] No up server in cluster >2013-12-12 14:21:04,203 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-60) [44ea9dad] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:21:57,211 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [14c65968] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,$clusterName myCluster,ACTION_TYPE_FAILED_NO_UP_SERVER_FOUND >2013-12-12 14:21:57,211 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-11) [14c65968] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 14:22:04,206 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-37) [72ccba09] No up server in cluster >2013-12-12 14:22:04,206 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-37) [72ccba09] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:22:04,207 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-37) [72ccba09] No up server in cluster >2013-12-12 14:22:04,207 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-37) [72ccba09] Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:22:05,828 WARN [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [7a1e2e57] CanDoAction of action StopGlusterVolume failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__GLUSTER_VOLUME,$volumeName StartMigrationDuringRebalanceTest,$vdsGroup myCluster,$clusterName myCluster,ACTION_TYPE_FAILED_NO_UP_SERVER_FOUND >2013-12-12 14:22:05,828 INFO [org.ovirt.engine.core.bll.gluster.StopGlusterVolumeCommand] (ajp-/127.0.0.1:8702-10) [7a1e2e57] Lock freed to object EngineLock [exclusiveLocks= key: 40302705-a79d-416e-b099-68670bf5637f value: GLUSTER >, sharedLocks= ] >2013-12-12 14:23:04,209 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-34) No up server in cluster >2013-12-12 14:23:04,209 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-34) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2013-12-12 14:23:04,210 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-34) No up server in cluster >2013-12-12 14:23:04,210 ERROR [org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob] (DefaultQuartzScheduler_Worker-34) Error updating tasks from CLI: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: NO_UP_SERVER_FOUND (Failed with error NO_UP_SERVER_FOUND and code 7000) > at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:30) [bll.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:83) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1041857
:
836010
|
836012
| 836013