Bug 1326262 - [userinterface_public_602]Metrics charts for pod are truncated intermittently
Summary: [userinterface_public_602]Metrics charts for pod are truncated intermittently
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Management Console
Version: 3.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Fabiano Franz
QA Contact: Yadan Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-12 09:39 UTC by Yanping Zhang
Modified: 2016-09-19 13:52 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-19 13:52:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
metrics-on-chrome (179.18 KB, image/png)
2016-04-12 09:42 UTC, Yanping Zhang
no flags Details
metrics-on-safari (289.55 KB, image/png)
2016-04-12 09:43 UTC, Yanping Zhang
no flags Details

Description Yanping Zhang 2016-04-12 09:39:56 UTC
Description of problem:
After deploy metrics, check pod's metrics on pod page on Safari 9 on Mac, when choose Time Range "Last 30 minutes", the charts for memory/cpu/network are shown with some blanks, it looks like being truncated. There is no this issue on Chrome 48,Firefox45, IE11


Version-Release number of selected component (if applicable):
fork_ami_openshift3_userinterface_console_1.3_210
openshift v1.1.6-73-ga2f457f
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5


How reproducible:
Always

Steps to Reproduce:
1.Deploy metrics on openshift, check pod's metrics on pod page on Safari 9 on mac, and choose Time Range "Last 30 minutes".
2.
3.

Actual results:
1.The charts looked like being truncated, while they are continuous on other browsers.

Expected results:
2.The charts should be continuous, and should be consistent with that shown on other browsers.

Additional info:

Comment 1 Yanping Zhang 2016-04-12 09:42:21 UTC
Created attachment 1146296 [details]
metrics-on-chrome

Comment 2 Yanping Zhang 2016-04-12 09:43:35 UTC
Created attachment 1146299 [details]
metrics-on-safari

Comment 3 Samuel Padgett 2016-04-12 13:40:33 UTC
I don't think it's a Safari vs Chrome problem. There's simply not enough data from Hawkular to fill in this area of the chart. This seems to happen if the system is under too much load.

https://github.com/openshift/origin/issues/7679#issuecomment-190369428

Jessica, would you object to making the smallest time range one hour? It'd greatly reduce the chances of charts like this. It'd also allow us to use point.max - point.min for calculated usage rates for cumulative metrics, which would be much cleaner.

Comment 4 Jessica Forrester 2016-04-12 14:10:12 UTC
Hmm I suppose so for now, although i'd like to see us be able to have smaller time ranges in the future.  Assume we are dependent on metrics sampling more often for that to be possible?

Comment 5 Samuel Padgett 2016-04-12 14:12:21 UTC
The sampling is going to be increased to every 10s from every 30s, but if this is a problem when the system is under load, it might not matter.

Adding Matt. Matt, I believe this is the same underlying issue as origin #7679.

Comment 6 Jessica Forrester 2016-04-27 21:32:47 UTC
The sampling update is done, Sam should this be transferred to Matt related to the load issue?

Comment 7 Samuel Padgett 2016-06-03 16:15:46 UTC
Yangping, can you confirm you had the latest metrics template when you tested. The frequency of sampling was increased to every 10s. Try running

$ oc get rc/heapster -n openshift-infra -o yaml

and look for "--metric_resolution=10s" under the container command.

If not, we should try with the latest templates to check if you still see the issue.

Comment 8 Samuel Padgett 2016-06-06 11:18:38 UTC
Marking ON_QA. I haven't seen issues with the new metric resolution value. Yangping, please make sure you test with the latest metrics template.

Comment 9 Yanping Zhang 2016-06-12 06:24:27 UTC
On devenv-rhel7_4354, tested with latest metrics template and images.
openshift/origin-metrics-cassandra                    latest              2aa439f8e002        3 hours ago         663.8 MB
openshift/origin-metrics-hawkular-metrics             latest              5549efe10a06        3 hours ago         770.7 MB
openshift/origin-metrics-heapster                     latest              4fcf7f02cb2a        3 hours ago         753.2 MB
openshift/origin-metrics-deployer                     latest              b7215d58ab95        3 hours ago         704.8 MB

$ oc get rc/heapster -n openshift-infra -o yaml
"--metric_resolution=10s" is under the container command.
Check the metrics on web, now the smallest Time Range is "Last hour", and charts are not truncated, the issue should have been fixed, so move the bug to Verified.


Note You need to log in before you can comment on or make changes to this bug.