Bug 1770779 - Some NodeFilesystem* alerts display wrong percentage
Summary: Some NodeFilesystem* alerts display wrong percentage
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.2.z
Assignee: Pawel Krupa
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On: 1770348
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-11 10:37 UTC by Pawel Krupa
Modified: 2019-12-03 22:43 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1770348
Environment:
Last Closed: 2019-12-03 22:43:15 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift cluster-monitoring-operator pull 540 'None' closed Bug 1770779: Fix filesystem alerts 2020-03-10 07:15:49 UTC
Red Hat Product Errata RHBA-2019:3953 None None None 2019-12-03 22:43:23 UTC

Description Pawel Krupa 2019-11-11 10:37:41 UTC
+++ This bug was initially created as a clone of Bug #1770348 +++

Description of problem:
Alert message value is not a percentage but is printed as one

Version-Release number of selected component (if applicable):
4.2.2

How reproducible:
Always

Steps to Reproduce:
1. Fill node filesystem to within alert levels
2. Observe alert message
3.

Actual results:
Filesystem on /dev/xvda3 at ip-10-0-143-157 has only 0.32% available space left and is filling up.

Expected results:
Filesystem on /dev/xvda3 at ip-10-0-143-157 has only 32% available space left and is filling up.

Additional info:
see attached alert message on web console

vs actually filesystem usage

[core@ip-10-0-143-157 ~]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        7.8G     0  7.8G   0% /dev
tmpfs           7.9G     0  7.9G   0% /dev/shm
tmpfs           7.9G  7.1M  7.9G   1% /run
tmpfs           7.9G     0  7.9G   0% /sys/fs/cgroup
/dev/xvda3       15G   11G  4.9G  68% /sysroot <--
/dev/xvda2      976M  144M  765M  16% /boot
tmpfs           1.6G     0  1.6G   0% /run/user/1000

--- Additional comment from Seth Jennings on 2019-11-08 19:42:41 UTC ---

Changes needed in these locations
https://github.com/openshift/cluster-monitoring-operator/blob/2389622ed26aee2bda2c0c58996013c6220603bb/assets/prometheus-k8s/rules.yaml#L438
https://github.com/openshift/cluster-monitoring-operator/blob/2389622ed26aee2bda2c0c58996013c6220603bb/assets/prometheus-k8s/rules.yaml#L455
https://github.com/openshift/cluster-monitoring-operator/blob/2389622ed26aee2bda2c0c58996013c6220603bb/assets/prometheus-k8s/rules.yaml#L471
https://github.com/openshift/cluster-monitoring-operator/blob/2389622ed26aee2bda2c0c58996013c6220603bb/assets/prometheus-k8s/rules.yaml#L485

To be like
https://github.com/openshift/cluster-monitoring-operator/blob/2389622ed26aee2bda2c0c58996013c6220603bb/assets/prometheus-k8s/rules.yaml#L500

--- Additional comment from Frederic Branczyk on 2019-11-11 10:25:44 UTC ---

Yes we should fix this, but we should use the `humanizePercentage` function available in the annotation templating, instead of fiddling with the conversion in PromQL, as this is a display concern.

--- Additional comment from Pawel Krupa on 2019-11-11 10:36:57 UTC ---

This is already fixed in 4.3 by doing multiplication by 100 (as seen in all links from https://bugzilla.redhat.com/show_bug.cgi?id=1770348#c1). 4.2 used a different format for those alerts.

Comment 4 errata-xmlrpc 2019-12-03 22:43:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3953


Note You need to log in before you can comment on or make changes to this bug.