Bug 1588436

Summary: Missing volume data when all nodes are down
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Filip Balák <fbalak>
Component: web-admin-tendrl-uiAssignee: Neha Gupta <negupta>
Status: CLOSED NOTABUG QA Contact: sds-qe-bugs
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rhgs-3.4CC: gshanmug, mbukatov, nthomas, rhs-bugs, sankarshan
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-13 07:51:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Empty volume list when machines are down
none
Cluster list with hidden volume but volume counter unchanged
none
Volume dashboard when nodes are down none

Description Filip Balák 2018-06-07 10:58:25 UTC 
Description of problem:
When all gluster nodes with a volume are shut down, UI data related to the volume is missing. The volume disappears from Volume page and in grafana are no data points.

Version-Release number of selected component (if applicable):
tendrl-ansible-1.6.3-4.el7rhgs.noarch
tendrl-api-1.6.3-3.el7rhgs.noarch
tendrl-api-httpd-1.6.3-3.el7rhgs.noarch
tendrl-commons-1.6.3-6.el7rhgs.noarch
tendrl-grafana-plugins-1.6.3-4.el7rhgs.noarch
tendrl-grafana-selinux-1.5.4-2.el7rhgs.noarch
tendrl-monitoring-integration-1.6.3-4.el7rhgs.noarch
tendrl-node-agent-1.6.3-6.el7rhgs.noarch
tendrl-notifier-1.6.3-3.el7rhgs.noarch
tendrl-selinux-1.5.4-2.el7rhgs.noarch
tendrl-ui-1.6.3-3.el7rhgs.noarch

How reproducible:
100%

Steps to Reproduce:
1. Import cluster with disperse volume.
2. Shut down all nodes.
3. Wait an hour.
4. Open Volumes page for given cluster.

Actual results:
The volume disappears from UI and there are no data related to the volume. In grafana are also no data points.

Expected results:
The volume is shown in UI and grafana as stopped.

Additional info:
Comment 1 gowtham 2018-06-12 09:02:50 UTC 
when all nodes are down TTL will delete the volume details from etcd, If all nodes are down we can't monitor the cluster. so there is no point in displaying volumes.
Comment 2 Filip Balák 2018-06-12 12:18:08 UTC 
Based on Comment 1 I close this as NOTABUG. Doc BZ 1590317 was created.
Comment 3 Martin Bukatovic 2018-06-12 12:40:53 UTC 
There is no clear summary backing closing decision, or references to particular
meeting.
Comment 4 Martin Bukatovic 2018-06-12 13:02:43 UTC 
When you monitor gluster trusted storage pool with RHGS WA, shutdown all
storage nodes and wait one hour, volumes hosted on these nodes will disappear
from RHGS WA ui and Grafana completely.

During "RHGS-3.4.0 In-flight bug triage" meeting on 2018-06-12, Nishanth pointed
out that this is not a bug, and the RHGS WA is expected to do this (see also
comment 1 from Gowtham).

Based on this, I'm asking Nishant to verify this quick summary, and to describe
how is RHGS WA expected to behave in such situation exactly instead, and then
to close the BZ.
Comment 5 Filip Balák 2018-06-12 13:06:13 UTC 
Created attachment 1450510 [details]
Empty volume list when machines are down
Comment 6 Filip Balák 2018-06-12 13:13:27 UTC 
Created attachment 1450513 [details]
Cluster list with hidden volume but volume counter unchanged
Comment 7 Filip Balák 2018-06-12 13:44:23 UTC 
Created attachment 1450520 [details]
Volume dashboard when nodes are down
Comment 8 Nishanth Thomas 2018-06-13 07:51:59 UTC 
Here is the expected behavior:

UI:

* Cluster unhealthy and all hosts marked as down
* No volumes and bricks listed
* events reflecting the relevant status

Dashboard:

* Host, volume, bricks panels reflects the relevant Counts updated(up, Down etc)
* Cluster/volume/brick status updated in all screens