Bug 1581060 - Metrics diagram can not be shown in project overview page
Summary: Metrics diagram can not be shown in project overview page
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.10.0
Assignee: John Sanda
QA Contact: Junqi Zhao
URL:
Whiteboard:
: 1581740 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-22 05:59 UTC by Junqi Zhao
Modified: 2018-07-30 19:16 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-30 19:16:14 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Metrics diagram can not be shown in overview page (184.72 KB, image/png)
2018-05-22 05:59 UTC, Junqi Zhao
no flags Details
Metrics diagram can be shown under "Metrics" tab of each pod (107.44 KB, image/png)
2018-05-22 06:00 UTC, Junqi Zhao
no flags Details
metrics pods log (31.72 KB, application/x-gzip)
2018-05-23 04:27 UTC, Junqi Zhao
no flags Details
Metrics diagram could be shown in overview page (126.33 KB, image/png)
2018-05-30 05:31 UTC, Junqi Zhao
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker HWKMETRICS-789 0 Blocker Resolved Optimized pod_id query causes NPE in SimpleTagQueryParser 2020-03-11 09:48:53 UTC
Red Hat Product Errata RHBA-2018:1816 0 None None None 2018-07-30 19:16:51 UTC

Description Junqi Zhao 2018-05-22 05:59:40 UTC
Created attachment 1439862 [details]
Metrics diagram can not be shown in overview page

Description of problem:
Metrics diagram can not be shown in overview page

Debugged the web page
"500 (Internal Server Error)" for 
https://hawkular-metrics.apps.0521-4oa.qe.rhcloud.com/hawkular/metrics/m/stats/query 

Metrics data could be shown under "Metrics" tab of each pod, see the attached picture

Version-Release number of selected component (if applicable):
# openshift version
openshift v3.10.0-0.50.0

metrics-cassandra/images/v3.10.0-0.50.0.0
metrics-heapster/images/v3.10.0-0.50.0.0
metrics-hawkular-metrics/images/v3.10.0-0.50.0.0
metrics-schema-installer/images/v3.10.0-0.50.0.0

How reproducible:
Always

Steps to Reproduce:
1. Deploy metrics 3.10 and login console/project/openshift-infra/overview page
2.
3.

Actual results:
Metrics diagram can not be shown in overview page

Expected results:
Metrics diagram could be shown in overview page

Additional info:

Comment 1 Junqi Zhao 2018-05-22 06:00:52 UTC
Created attachment 1439864 [details]
Metrics diagram can be shown under "Metrics" tab of each pod

Comment 2 John Sanda 2018-05-22 16:26:09 UTC
Hi Junqi,

When creating tickets can you please include the logs for heapster, cassandra, and hawkular-metrics? We usually will need them to investigate the issue, and this will eliminate the extra step of having to follow up and ask for them. Thanks.

Comment 3 Junqi Zhao 2018-05-23 04:27:45 UTC
Created attachment 1440410 [details]
metrics pods log

Comment 4 John Sanda 2018-05-23 15:33:27 UTC
here is a regression with the REST endpoint,

POST /hawkular/metrics/m/stats/query

which is used for rendering graphs for deployments, statefulsets, etc. It fails with an NPE in SimpleTagQueryParser. Here is the stack trace from a recent log:

2018-05-23 03:33:13,357 ERROR [org.hawkular.metrics.api.jaxrs.util.ApiUtils] (RxComputationScheduler-3) HAWKMETRICS200010: Failed to process request: java.lang.NullPointerException
        at org.hawkular.metrics.core.service.tags.SimpleTagQueryParser.lambda$tagValueFilter$40(SimpleTagQueryParser.java:557)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:67)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:76)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:395)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:355)
        at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:846)
        at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onNext(OnSubscribeDoOnEach.java:101)
        at rx.observers.Subscribers$5.onNext(Subscribers.java:235)
        at rx.internal.operators.OnSubscribeSwitchIfEmpty$ParentSubscriber.onNext(OnSubscribeSwitchIfEmpty.java:121)
        at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:395)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:355)
        at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:846)
        at org.hawkular.rx.cassandra.driver.ResultSetToRowsTransformer$RowProducer.produce(ResultSetToRowsTransformer.java:111)
        at org.hawkular.rx.cassandra.driver.ResultSetToRowsTransformer$RowProducer.lambda$execute$0(ResultSetToRowsTransformer.java:154)
        at rx.internal.schedulers.EventLoopsScheduler$EventLoopWorker$1.call(EventLoopsScheduler.java:172)
        at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: org.hawkular.metrics.model.Metric.class
        at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:71)
        ... 22 more

This is a new regression as a result of changes made in SimpleTagQueryParser for 3.10. It will prevent graphs from rendering.

Comment 5 John Sanda 2018-05-23 15:33:43 UTC
*** Bug 1581740 has been marked as a duplicate of this bug. ***

Comment 7 Junqi Zhao 2018-05-30 05:30:25 UTC
Please change to ON_QA, tested with metrics-hawkular-metrics:v3.10.0-0.54.0.0, issue is fixed

Comment 8 Junqi Zhao 2018-05-30 05:31:20 UTC
Created attachment 1445634 [details]
Metrics diagram could be shown in overview page

Comment 9 Junqi Zhao 2018-05-31 00:07:20 UTC
Set it to VERIFIED as per Comment 7 and Comment 8

Comment 11 errata-xmlrpc 2018-07-30 19:16:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816


Note You need to log in before you can comment on or make changes to this bug.