Bug 1664180 - most of the metrics diagrams are not shown under "Administration -> Nodes" on cluster console page
Summary: most of the metrics diagrams are not shown under "Administration -> Nodes" on...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.1.0
Assignee: Frederic Branczyk
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-08 00:43 UTC by Junqi Zhao
Modified: 2019-06-04 10:41 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:41:43 UTC
Target Upstream Version:


Attachments (Terms of Use)
metrics diagrams are not shown under "Administration -> Nodes" (426.00 KB, image/png)
2019-01-08 00:43 UTC, Junqi Zhao
no flags Details
there is not Memory diagram for node (204.77 KB, image/png)
2019-02-19 10:41 UTC, Junqi Zhao
no flags Details
Memory diagram is shown for node (194.15 KB, image/png)
2019-02-26 01:41 UTC, Junqi Zhao
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:41:49 UTC

Description Junqi Zhao 2019-01-08 00:43:29 UTC
Created attachment 1519098 [details]
metrics diagrams are not shown under "Administration -> Nodes"

Description of problem:
This bug is cloned from https://jira.coreos.com/browse/MON-508
File it again for QE team to track the monitoring issue in Bugzilla.

Login cluster console with admin user, and click "Administration -> Nodes", select one node to check the "Node Overview" part, besides "Number of Pods" has diagram, other parts don't have diagrams.

 

Errors:

Graph error: No data from query for RAM.
Graph error: No data from query for CPU.
Graph error: No data from query for Filesystem.
Graph error: No data from query for Network In.
Graph error: No data from query for Network Out.

And

GET https://console-openshift-console.apps.juzhao.origin-ci-int-aws.dev.rhcloud.com/api/kubernetes/api/v1/namespaces/openshift-monitoring/configmaps/sharing-config 404 (Not Found)

 

sharing-config configmap is not exist

# oc -n openshift-monitoring get cm
NAME                                        DATA      AGE
adapter-config                              1         6h
cluster-monitoring-config                   1         6h
grafana-dashboard-k8s-cluster-rsrc-use      1         6h
grafana-dashboard-k8s-node-rsrc-use         1         6h
grafana-dashboard-k8s-resources-cluster     1         6h
grafana-dashboard-k8s-resources-namespace   1         6h
grafana-dashboard-k8s-resources-pod         1         6h
grafana-dashboards                          1         6h
prometheus-adapter-prometheus-config        1         6h
prometheus-k8s-rulefiles-0                  1         6h
serving-certs-ca-bundle                     1         6h

Version-Release number of selected component (if applicable):
docker.io/grafana/grafana:5.2.4
docker.io/openshift/oauth-proxy:v1.1.0
docker.io/openshift/prometheus-alertmanager:v0.15.2
docker.io/openshift/prometheus-node-exporter:v0.16.0
docker.io/openshift/prometheus:v2.5.0
quay.io/coreos/configmap-reload:v0.0.1
quay.io/coreos/k8s-prometheus-adapter-amd64:v0.4.1
quay.io/coreos/kube-rbac-proxy:v0.4.0
quay.io/coreos/kube-state-metrics:v1.4.0
quay.io/coreos/prom-label-proxy:v0.1.0
quay.io/coreos/prometheus-config-reloader:v0.26.0
quay.io/coreos/prometheus-operator:v0.26.0
quay.io/openshift-release-dev/ocp-v4.0@sha256:5c4abcf8e45bd9a79d10bc837d17c004d5670ae7081f0f3b835c6a1c5ad4dfda

How reproducible:
Always

Steps to Reproduce:
1. Login cluster console with admin user, and click "Administration -> Nodes", select one node to check the "Node Overview" part
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Junqi Zhao 2019-01-16 02:02:43 UTC
anaylysis

RAM
expression: node_memory_Active{instance=~'.*${node}.*'}
failed reason:
missed following rule
record: node_memory_Active expr: node_memory_Active_bytes

CPU
expression: instance:node_cpu:rate:sum{instance=~'.*${node}.*'}
failed reason:
missed following rule
record: instance:node_cpu:rate:sum expr: sum
  by(instance) (rate(node_cpu{mode!="idle",mode!="iowait"}[3m]))

Network In
expression: instance:node_network_receive_bytes:rate:sum{instance=~'.*${node}.*'}
failed reason:
missed following rule
record: instance:node_network_receive_bytes:rate:sum expr: sum
  by(instance) (rate(node_network_receive_bytes[3m]))


Network Out
expression: instance:node_network_transmit_bytes:rate:sum{instance=~'.*${node}.*'}
failed reason:
missed following rule
record: instance:node_network_transmit_bytes:rate:sum expr: sum
  by(instance) (rate(node_network_transmit_bytes[3m]))



Filesystem
expression: instance:node_filesystem_usage:sum{instance=~'.*${node}.*'}
failed reason:
missed following rule
record: instance:node_filesystem_usage:sum expr: sum
  by(instance) ((node_filesystem_size{mountpoint="/"} - node_filesystem_free{mountpoint="/"}))

Comment 2 Frederic Branczyk 2019-02-06 12:33:54 UTC
Moving this to modified, just like the respective jira issue already is.

Comment 4 Junqi Zhao 2019-02-19 04:05:38 UTC
Blocked by regression Bug 1678310

Comment 6 Junqi Zhao 2019-02-19 10:41:27 UTC
Created attachment 1536284 [details]
there is not Memory diagram for node

Comment 7 Andrew Pickering 2019-02-25 09:05:03 UTC
Node memory graph fixed by PR https://github.com/openshift/console/pull/1190, which merged today.

Comment 8 Junqi Zhao 2019-02-26 01:40:44 UTC
ssue is fixed with

$ oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.0.0-0.nightly-2019-02-25-194625 True False 7m36s Cluster version is 4.0.0-0.nightly-2019-02-25-194625

RHCOS build: 47.330

Comment 9 Junqi Zhao 2019-02-26 01:41:28 UTC
Created attachment 1538641 [details]
Memory diagram is shown for node

Comment 12 errata-xmlrpc 2019-06-04 10:41:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.