I see the following errors in engine.log, when there are no RHS Nodes are UP in the gluster cluster 015-04-17 05:00:17,148 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) Failed in GlusterServersListVDS method 2015-04-17 05:00:17,149 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GlusterServersListVDSCommand(HostName = dhcp37-150.lab.eng.blr.redhat.com, HostId = c3c48853-7af8-4bb1-8599-caa6839df7cf) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GlusterServersListVDS, error = Command execution failed error: Connection failed. Please check if gluster daemon is operational. peer status: failed return code: 1, code = -32603 2015-04-17 05:00:17,149 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, log id: 12dbd7ba 2015-04-17 05:00:17,149 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-16) Error while refreshing server data for cluster gluster-cluster from database!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GlusterServersListVDS, error = Command execution failed error: Connection failed. Please check if gluster daemon is operational. peer status: failed return code: 1, code = -32603 (Failed with error unexpected and code 16) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.fetchServers(GlusterJob.java:55) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchServers(GlusterSyncJob.java:319) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchServers(GlusterSyncJob.java:260) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshServerData(GlusterSyncJob.java:137) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:107) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) [:1.7.0_79] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_79] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_79] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Again this error message is flooding the engine.log Any thoughts ?
Tested with vt14.3 I am not seeing the error message when there are no RH-Gluster Server Nodes available in gluster enabled cluster. I seeing INFO messages though, 2015-04-17 05:14:44,266 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster 2015-04-17 05:14:44,268 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster Based on the above observation, verifying this bug
I wish to have seperate discussion in my comment5 When one of the RH-Gluster Storage Server available in gluster cluster is found dead or not UP due to different conditions, then that event is logged continuously in bulk for every second or so. ( as seen in comment5 ) This blows up the log file. I will initiate a seperate bug for this issue, if its really agreed its a issue
removing needinfo on sahina as because of comment7
If this bug requires doc text for errata release, please provide draft text in the doc text field in the following format: Cause: Consequence: Fix: Result: The documentation team will review, edit, and approve the text. If this bug does not require doc text, please set the 'requires_doc_text' flag to -.
Giving requires_doc_text to "-" as this is not a visible issue to customer. was an issue regarding a redundant flooding of logs.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0888.html