Bug 1475681

Summary: [RFE][Tracker] Add initial Kibana dashboard
Product: [oVirt] ovirt-engine-metrics Reporter: Shirly Radco <sradco>
Component: RFEsAssignee: Shirly Radco <sradco>
Status: CLOSED CURRENTRELEASE QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: high    
Version: 1.0.5CC: bugs, lsvaty, pstehlik, sradco, ylavi
Target Milestone: ovirt-4.2.3Keywords: FutureFeature
Target Release: 1.1.4.2Flags: rule-engine: ovirt-4.2+
rule-engine: blocker+
pstehlik: testing_plan_complete-
ylavi: planning_ack+
rule-engine: devel_ack+
pstehlik: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-metrics-1.1.4.2-1.el7ev Doc Type: Enhancement
Doc Text:
Feature: Example dashboards have been added. Reason: To help users learn hows to create their own dashboards faster, by examples. Result: A directory has been added to /etc/ovirt-engine-metrics/ with the example dashboards and visualizations. They need to be copied to the local machine where the user views Kibana and uploaded using the Kibana UI.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-31 09:38:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Metrics RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1402901, 1465012, 1513026, 1583158    

Description Shirly Radco 2017-07-27 07:30:41 UTC
Description of problem:
Create a kibana dashboard to show the general health of the oVirt environment.

Comment 1 Yaniv Lavi 2017-07-31 09:04:29 UTC
Do we have a way to push a dashboard into Kibana externally in the viaq setup?

Comment 2 Shirly Radco 2017-07-31 09:33:27 UTC
According to what Rich said it will be possible in OpenShift 3.6.

Comment 3 Lukas Svaty 2017-08-18 08:03:42 UTC
Hi these would be few of the dashboard queries it would be interesting to see.

- some efficient queries from DWH report based on timeline (what was status of - DC, storage, VMs, hosts at specific time)
- API response avarage/actual times graph
- trend graph of disk space per storage / per DC / per cluster (future expectation of 1 month)
- parsable engine and vdsm logs based on Correlation-ID to easy seach cause of problem

Comment 4 Yaniv Kaul 2018-01-11 13:34:20 UTC
Is that realistic to get into 4.2.1?

Comment 6 Lukas Svaty 2018-04-19 13:33:13 UTC
Error while importing visualizations:

Error: Could not locate that index-pattern-field (id: ovirt.engine_fqdn.raw)
KbnError@https://kibana.example.com/bundles/commons.bundle.js?v=10229:57511:21
SavedObjectNotFound@https://kibana.example.com/bundles/commons.bundle.js?v=10229:57640:6
FieldAggParamFactory/FieldAggParam.prototype.deserialize@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:49376:16
AggConfigFactory/AggConfig.prototype.fillDefaults/<@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:53866:20
AggConfigFactory/AggConfig.prototype.fillDefaults@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:53844:8
AggConfig@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:53758:8
AggConfigs/<.initialSet<@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:54464:19
AggConfigs@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:54462:22
VisFactory/Vis.prototype.setState@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:48822:20
__WEBPACK_AMD_DEFINE_RESULT__</</SavedVis.prototype._updateVis@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:56374:8
__WEBPACK_AMD_DEFINE_RESULT__</</SavedVis.prototype._afterEsResp/<@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:56317:28
processQueue@https://kibana.example.com/bundles/commons.bundle.js?v=10229:42452:29
scheduleProcessQueue/<@https://kibana.example.com/bundles/commons.bundle.js?v=10229:42468:28
$RootScopeProvider/this.$get</Scope.prototype.$eval@https://kibana.example.com/bundles/commons.bundle.js?v=10229:43696:17
$RootScopeProvider/this.$get</Scope.prototype.$digest@https://kibana.example.com/bundles/commons.bundle.js?v=10229:43507:16
$RootScopeProvider/this.$get</Scope.prototype.$apply@https://kibana.example.com/bundles/commons.bundle.js?v=10229:43804:14
link/</reader.onload@https://kibana.example.com/bundles/kibana.bundle.js?v=10229:62025:14

Comment 7 Red Hat Bugzilla Rules Engine 2018-04-19 13:33:20 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 8 Shirly Radco 2018-04-23 07:25:52 UTC
I added a step to the kibana dashboards import

https://github.com/sradco/ovirt-site/blob/aa2d653c543eb51571b22455ba1e497032971c7a/source/develop/release-management/features/metrics/metrics-store-installation.html.md

In order for field of type raw to be accessed, the index should be refreshed before loading the visualizations.
They are generated automatically when a sting fields are saved to the index.

Comment 9 Lukas Svaty 2018-04-24 06:56:43 UTC
Same result after the refresh of indices:


Saved Objects: Could not locate that index-pattern (id: project.ovirt-metrics-test-engine.d3a9f511-0508-11e8-9a49-001a4a013f0b.*)

Error: Could not locate that index-pattern (id: project.ovirt-metrics-test-engine.d3a9f511-0508-11e8-9a49-001a4a013f0b.*)
KbnError@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:57511:21
SavedObjectNotFound@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:57640:6
applyESResp@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/kibana.bundle.js?v=10229:45045:37
processQueue@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:42452:29
scheduleProcessQueue/<@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:42468:28
$RootScopeProvider/this.$get</Scope.prototype.$eval@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:43696:17
$RootScopeProvider/this.$get</Scope.prototype.$digest@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:43507:16
$RootScopeProvider/this.$get</Scope.prototype.$apply@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:43804:14
done@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:38253:37
completeRequest@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:38451:8
requestLoaded@https://kibana.tlv-metrics-store.eng.lab.tlv.redhat.com/bundles/commons.bundle.js?v=10229:38392:10

Comment 10 Yaniv Kaul 2018-04-24 10:21:48 UTC
Target milestone is 4.2.2?

Comment 11 Shirly Radco 2018-04-24 13:07:56 UTC
(In reply to Yaniv Kaul from comment #10)
> Target milestone is 4.2.2?

I see I moved it to 4.2.3 by mistake. Moved back to 4.2.3.

Comment 13 Red Hat Bugzilla Rules Engine 2018-04-27 08:09:20 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 15 Lukas Svaty 2018-05-03 12:49:02 UTC
Yes, all indices were refreshed.

Comment 16 Shirly Radco 2018-05-14 07:02:40 UTC
Please make sure that you wait a minute and make sure that collectd and fluentd are running on the host with the vms. If the data about vms is not collected it can't be used in the widgets/dashboards.

Comment 17 Lukas Svaty 2018-05-14 14:12:50 UTC
What if there are no hosts/no Vms in the engine?

Comment 18 Shirly Radco 2018-05-14 18:35:40 UTC
Kibana only works with existing fields in the indexes.
If the field is not in the index you will not be able to import the widgets using it and they will have an error in the dashboard for the specific missing widget.

Comment 19 Lukas Svaty 2018-05-16 14:45:53 UTC
tried with production engine, ~8 hosts, few hundreds of VMs

Error: Could not locate that index-pattern-field (id: ovirt.engine_fqdn.raw)

Please schedule a meeting with me and I can show you this issue, if you require some additional logs please let me know which ones.

Comment 20 Lukas Svaty 2018-05-30 08:48:25 UTC
Verified after consultation with DEV on BRQ production environment

Version: ovirt-engine-metrics-1.1.4.2-1.el7ev

Needed to rerun playbook at least with --scope-engine

Comment 21 Sandro Bonazzola 2018-05-31 09:38:15 UTC
This bugzilla is included in oVirt 4.2.3 release, published on May 4th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.3 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.