Bug 1581060

Summary: Metrics diagram can not be shown in project overview page
Product: OpenShift Container Platform Reporter: Junqi Zhao <juzhao>
Component: HawkularAssignee: John Sanda <jsanda>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.10.0CC: aos-bugs, jsanda, juzhao, rvargasp
Target Milestone: ---Keywords: Regression
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-30 19:16:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Metrics diagram can not be shown in overview page
none
Metrics diagram can be shown under "Metrics" tab of each pod
none
metrics pods log
none
Metrics diagram could be shown in overview page none

Description Junqi Zhao 2018-05-22 05:59:40 UTC
Created attachment 1439862 [details]
Metrics diagram can not be shown in overview page

Description of problem:
Metrics diagram can not be shown in overview page

Debugged the web page
"500 (Internal Server Error)" for 
https://hawkular-metrics.apps.0521-4oa.qe.rhcloud.com/hawkular/metrics/m/stats/query 

Metrics data could be shown under "Metrics" tab of each pod, see the attached picture

Version-Release number of selected component (if applicable):
# openshift version
openshift v3.10.0-0.50.0

metrics-cassandra/images/v3.10.0-0.50.0.0
metrics-heapster/images/v3.10.0-0.50.0.0
metrics-hawkular-metrics/images/v3.10.0-0.50.0.0
metrics-schema-installer/images/v3.10.0-0.50.0.0

How reproducible:
Always

Steps to Reproduce:
1. Deploy metrics 3.10 and login console/project/openshift-infra/overview page
2.
3.

Actual results:
Metrics diagram can not be shown in overview page

Expected results:
Metrics diagram could be shown in overview page

Additional info:

Comment 1 Junqi Zhao 2018-05-22 06:00:52 UTC
Created attachment 1439864 [details]
Metrics diagram can be shown under "Metrics" tab of each pod

Comment 2 John Sanda 2018-05-22 16:26:09 UTC
Hi Junqi,

When creating tickets can you please include the logs for heapster, cassandra, and hawkular-metrics? We usually will need them to investigate the issue, and this will eliminate the extra step of having to follow up and ask for them. Thanks.

Comment 3 Junqi Zhao 2018-05-23 04:27:45 UTC
Created attachment 1440410 [details]
metrics pods log

Comment 4 John Sanda 2018-05-23 15:33:27 UTC
here is a regression with the REST endpoint,

POST /hawkular/metrics/m/stats/query

which is used for rendering graphs for deployments, statefulsets, etc. It fails with an NPE in SimpleTagQueryParser. Here is the stack trace from a recent log:

2018-05-23 03:33:13,357 ERROR [org.hawkular.metrics.api.jaxrs.util.ApiUtils] (RxComputationScheduler-3) HAWKMETRICS200010: Failed to process request: java.lang.NullPointerException
        at org.hawkular.metrics.core.service.tags.SimpleTagQueryParser.lambda$tagValueFilter$40(SimpleTagQueryParser.java:557)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:67)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:76)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:395)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:355)
        at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:846)
        at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onNext(OnSubscribeDoOnEach.java:101)
        at rx.observers.Subscribers$5.onNext(Subscribers.java:235)
        at rx.internal.operators.OnSubscribeSwitchIfEmpty$ParentSubscriber.onNext(OnSubscribeSwitchIfEmpty.java:121)
        at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:395)
        at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:355)
        at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:846)
        at org.hawkular.rx.cassandra.driver.ResultSetToRowsTransformer$RowProducer.produce(ResultSetToRowsTransformer.java:111)
        at org.hawkular.rx.cassandra.driver.ResultSetToRowsTransformer$RowProducer.lambda$execute$0(ResultSetToRowsTransformer.java:154)
        at rx.internal.schedulers.EventLoopsScheduler$EventLoopWorker$1.call(EventLoopsScheduler.java:172)
        at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: org.hawkular.metrics.model.Metric.class
        at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
        at rx.internal.operators.OnSubscribeFilter$FilterSubscriber.onNext(OnSubscribeFilter.java:71)
        ... 22 more

This is a new regression as a result of changes made in SimpleTagQueryParser for 3.10. It will prevent graphs from rendering.

Comment 5 John Sanda 2018-05-23 15:33:43 UTC
*** Bug 1581740 has been marked as a duplicate of this bug. ***

Comment 7 Junqi Zhao 2018-05-30 05:30:25 UTC
Please change to ON_QA, tested with metrics-hawkular-metrics:v3.10.0-0.54.0.0, issue is fixed

Comment 8 Junqi Zhao 2018-05-30 05:31:20 UTC
Created attachment 1445634 [details]
Metrics diagram could be shown in overview page

Comment 9 Junqi Zhao 2018-05-31 00:07:20 UTC
Set it to VERIFIED as per Comment 7 and Comment 8

Comment 11 errata-xmlrpc 2018-07-30 19:16:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816