Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1209131

Summary: "VdcBLLException: NO_UP_SERVER_FOUND" in seen in engine logs
Product: Red Hat Enterprise Virtualization Manager Reporter: rhev-integ
Component: ovirt-engineAssignee: Sahina Bose <sabose>
Status: CLOSED ERRATA QA Contact: SATHEESARAN <sasundar>
Severity: high Docs Contact:
Priority: medium    
Version: 3.4.3-1CC: amureini, bugs, dpati, dtsang, ecohen, eedri, gklein, iheim, juwu, kmayilsa, knarra, lpeer, lsurette, mgoldboi, mkalinin, mmahoney, pagupta, pprakash, pstehlik, rbalakri, Rhev-m-bugs, rhs-bugs, sabose, sdharane, ssampat, yeylon, ylavi
Target Milestone: ---Keywords: ZStream
Target Release: 3.5.1Flags: ylavi: Triaged+
Hardware: x86_64   
OS: Linux   
Whiteboard: gluster
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1191494 Environment:
Last Closed: 2015-04-28 18:49:40 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1191494    
Bug Blocks:    

Comment 5 SATHEESARAN 2015-04-17 08:47:34 UTC
I see the following errors in engine.log, when there are no RHS Nodes are UP in the gluster cluster

015-04-17 05:00:17,148 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) Failed in GlusterServersListVDS method
2015-04-17 05:00:17,149 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GlusterServersListVDSCommand(HostName = dhcp37-150.lab.eng.blr.redhat.com, HostId = c3c48853-7af8-4bb1-8599-caa6839df7cf) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GlusterServersListVDS, error = Command execution failed
error: Connection failed. Please check if gluster daemon is operational.
peer status: failed
return code: 1, code = -32603
2015-04-17 05:00:17,149 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GlusterServersListVDSCommand, log id: 12dbd7ba
2015-04-17 05:00:17,149 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-16) Error while refreshing server data for cluster gluster-cluster from database!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GlusterServersListVDS, error = Command execution failed
error: Connection failed. Please check if gluster daemon is operational.
peer status: failed
return code: 1, code = -32603 (Failed with error unexpected and code 16)
        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterJob.fetchServers(GlusterJob.java:55) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchServers(GlusterSyncJob.java:319) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchServers(GlusterSyncJob.java:260) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshServerData(GlusterSyncJob.java:137) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:107) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) [:1.7.0_79]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_79]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_79]
        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]



Again this error message is flooding the engine.log

Any thoughts ?

Comment 6 SATHEESARAN 2015-04-17 09:00:58 UTC
Tested with vt14.3
I am not seeing the error message when there are no RH-Gluster Server Nodes available in gluster enabled cluster.

I seeing INFO messages though,
2015-04-17 05:14:44,266 INFO  [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster
2015-04-17 05:14:44,268 INFO  [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler_Worker-95) No up server in cluster


Based on the above observation, verifying this bug

Comment 7 SATHEESARAN 2015-04-17 09:03:38 UTC
I wish to have seperate discussion in my comment5

When one of the RH-Gluster Storage Server available in gluster cluster is found dead or not UP due to different conditions, then that event is logged continuously in bulk for every second or so. ( as seen in comment5 )
This blows up the log file.

I will initiate a seperate bug for this issue, if its really agreed its a issue

Comment 8 SATHEESARAN 2015-04-17 09:04:29 UTC
removing needinfo on sahina as because of comment7

Comment 9 Julie 2015-04-20 05:01:07 UTC
If this bug requires doc text for errata release, please provide draft text in the doc text field in the following format:

Cause:
Consequence:
Fix:
Result:

The documentation team will review, edit, and approve the text.

If this bug does not require doc text, please set the 'requires_doc_text' flag to -.

Comment 10 Sahina Bose 2015-04-20 05:43:02 UTC
Giving requires_doc_text to "-" as this is not a visible issue to customer. was an issue regarding a redundant flooding of logs.

Comment 11 errata-xmlrpc 2015-04-28 18:49:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0888.html