Bug 1507942 - Weird line in Memory Utilization chart when stressed
Summary: Weird line in Memory Utilization chart when stressed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: web-admin-tendrl-monitoring-integration
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Ankush Behl
QA Contact: Martin Kudlej
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-31 14:12 UTC by Filip Balák
Modified: 2017-12-18 04:39 UTC (History)
6 users (show)

Fixed In Version: tendrl-monitoring-integration-1.5.4-3.el7rhgs
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-18 04:39:36 UTC
Target Upstream Version:


Attachments (Terms of Use)
Memory Utilization chart (89.67 KB, image/png)
2017-10-31 14:12 UTC, Filip Balák
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:3478 normal SHIPPED_LIVE RHGS Web Administration packages 2017-12-18 09:34:49 UTC
Github https://github.com/Tendrl monitoring-integration issues 230 None None None 2017-11-07 07:03:33 UTC

Description Filip Balák 2017-10-31 14:12:39 UTC
Created attachment 1345940 [details]
Memory Utilization chart

Description of problem:
When I run stress to increase memory consumption to be more than 80% then there appears strange orange line in Memory Utilization chart in Hosts dashboard. This line does not have any label and orientation of its filling is up (other lines have some filling from 0% to the value that represents them). The orange line doesn't reflect cached memory. For that is used another line that is visualized correctly.

Version-Release number of selected component (if applicable):
tendrl-api-httpd-1.5.3-2.el7rhgs.noarch
tendrl-grafana-selinux-1.5.3-2.el7rhgs.noarch
tendrl-selinux-1.5.3-2.el7rhgs.noarch
tendrl-node-agent-1.5.3-3.el7rhgs.noarch
tendrl-ui-1.5.3-2.el7rhgs.noarch
tendrl-grafana-plugins-1.5.3-2.el7rhgs.noarch
tendrl-notifier-1.5.3-1.el7rhgs.noarch
tendrl-ansible-1.5.3-2.el7rhgs.noarch
tendrl-commons-1.5.3-1.el7rhgs.noarch
tendrl-api-1.5.3-2.el7rhgs.noarch
tendrl-monitoring-integration-1.5.3-2.el7rhgs.noarch
glusterfs-3.8.4-50.el7rhgs.x86_64

How reproducible:
80%

Steps to Reproduce:
1. Import cluster with volume.
2. Stress the memory consumption above 80%. E.g.:
   stress --vm-bytes $(awk '/MemFree/{printf "%d\n", $2 * 2;}' < /proc/meminfo)k --vm-keep -m 1
3. Check Memory Utilization chart in Hosts dashboard.

Actual results:
There appears orange line with no label.

Expected results:
There should be no line that is not described in chart legend.

Additional info:

Comment 1 Ankush Behl 2017-11-02 10:14:28 UTC
@fbalak@redhat.com The orange color line is a threshold line, which is set to 80%. If a threshold is increased by 80% the warning will be raised which then will be shown in tendrl-ui.

Comment 2 Filip Balák 2017-11-02 10:45:13 UTC
Ok, the line seemed to me little odd because there was no tooltip and the colour is almost the same as colour for cached memory but this is probably not a bug. It would be good to change the colour to red and decrease its opacity if possible. Maybe it would be also good to add a tooltip for this line that says something like "Warning - Almost out of memory".

Comment 3 Martin Kudlej 2017-11-02 12:02:38 UTC
I think this is bug because it should be clear what is it.

Comment 4 Ankush Behl 2017-11-04 20:37:13 UTC
@julim@redhat.com can you provide a proper tooltip message for the warning threshold bar.

Comment 5 Ankush Behl 2017-11-04 21:31:15 UTC
@fbalak@redhat.com I tried using the red color bar with low opacity but it still feels the same because this time the "used" memory bar in memory panel seems to be conflicting(both are getting red). So I think we can add a tooltip to the panel which can clearly state that there is a warning threshold bar set at 80%.

Comment 6 Petr Penicka 2017-11-08 14:13:22 UTC
Triage Nov 8: pending Ju's input.

Comment 7 Ju Lim 2017-11-08 19:13:19 UTC
@anbehl@redhat.com

How about the following tooltip: "Memory utilization has exceeded the 80% warning threshold"?

Comment 8 Ankush Behl 2017-11-08 19:29:46 UTC
@julim@redhat.com Sorry I forgot to mention the context for tool-tip. So we need tool-tip that specifies user that we have a threshold bar set at 80% so don't get confused.

In this case, the tool-tip you mentioned will not be an apt tool-tip I think.

Comment 9 Ju Lim 2017-11-08 19:46:20 UTC
Try "The memory utilization threshold is set to >= 80% and is displayed as an orange area in the 80-100% range."

Comment 10 Ankush Behl 2017-11-08 19:53:51 UTC
@rghatvis@redhat.com can you review the text suggested by @julim@redhat.com and provide your inputs on it.

Comment 11 Ju Lim 2017-11-08 19:55:39 UTC
Alternatively, try "Values in the 80-100% range exceed the memory utilization threshold" to be less wordy, but @rghatvis@redhat.com can probably provide better guidance on the tooltip.

Is it possible instead of doing an orange area for 80-100% to only show a orange line at 80% as the area does seem to be confusing.  It's not typical to see the area shaded (unless it's a dynamic threshold) especially since the threshold is set statically set to 80% (and it does not vary).

Comment 13 Rakesh 2017-11-08 22:11:27 UTC
(In reply to Ankush Behl from comment #10)
> @rghatvis@redhat.com can you review the text suggested by @julim@redhat.com
> and provide your inputs on it.

Ankush, 
Both the versions suggested by Ju looks good to me. I expanded a bit(not succinct this time):

"The memory utilization threshold is set to 80%. Reaching or exceeding this value displays an orange horizontal line indicating high memory usage."

We don't need to mention about the orange shaded area(not helpful to the user) and it would be nice if we can take it off from the UI as Ju suggested.

Ju, correct me if I am wrong, aren't tooltips used for supplementary information? Info that is not super important to the user? In my knowledge yes, it is. Then why are we considering a "tooltip" to warn or alert the user about high memory consumption? This is a critical alert so instead of a tooltip where the user needs to hover over the icon, how about a pop-up box popping out to warn the user?

Comment 14 Ankush Behl 2017-11-10 06:57:08 UTC
Added Tool-tip which is provided by @rghatvis@redhat.com

Comment 15 Ju Lim 2017-11-13 21:24:24 UTC
@rghatvis@redhat.com We have to work within the constraints of what the tool (i.e. Grafana) allows.  Ankush thinks it's possible to just do a line (vs. the shaded area) for the static threshold.  It is not uncommon that as you hover over different things in the chart that the tooltip will give you additional / supplementary information indicating what it is.

A pop-up box would be annoying to a user especially when you're looking at a "dashboard" which may contain multiple threshold breaches, and it will quickly very noisy and potentially annoying if lots of pop-ups sprung up all over, which require user to click to close out the pop-up.

There are other treatments that can be done, but probably beyond the scope for what is needed at present, e.g. remediation, correlation with the root cause of the event that caused the breach, etc.

Comment 17 Bala Konda Reddy M 2017-11-18 12:49:20 UTC
Verified with tendrl-monitoring-integration-1.5.4-3

Imported the cluster. stressed one storage node as mentioned in the description.

For Memory Utilization widget, the tool-tip says the following.
 
"The memory utilization threshold is set to 80%. Reaching or exceeding this value displays an orange horizontal line indicating high memory usage." as suggested from the previous comments.

The explanation for the line at 80% is given and it is appropriate. After 80% of used memory, there is a orange shaded region 80-100 for specifying for maximum memory utilization

Hence marking it as verified

Comment 20 errata-xmlrpc 2017-12-18 04:39:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3478


Note You need to log in before you can comment on or make changes to this bug.