Bug 1494912
| Summary: | metrics indexes should not save the _source | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Shirly Radco <sradco> | ||||||
| Component: | Logging | Assignee: | Rich Megginson <rmeggins> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Lukas Svaty <lsvaty> | ||||||
| Severity: | low | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 3.6.1 | CC: | aos-bugs, jcantril, lsvaty, lvlcek, rmeggins, sradco, wsun | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 3.7.z | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: |
Cause: The _source was not disabled for metrics records being stored in Elasticsearch.
Consequence: The records were taking up much more resources (cpu, ram, disk) than necessary.
Fix: Completely disable the _source for project.ovirt-metrics* records.
Result: Metrics records are much smaller and require fewer resources to handle.
|
Story Points: | --- | ||||||
| Clone Of: | |||||||||
| : | 1512132 (view as bug list) | Environment: | |||||||
| Last Closed: | 2017-12-18 13:22:48 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1512132 | ||||||||
| Attachments: |
|
||||||||
Created attachment 1330073 [details]
kibana_screenshot2
The source is not saved for the collectd fields. Has something changed? (In reply to Rich Megginson from comment #2) > The source is not saved for the collectd fields. > > Has something changed? _source and _all fields should all be disabled for the the records in the indexes of project.ovirt-metrics* should be removed. As recommended by https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html Currently only collectd namespaces have source disabled. The fields are enriched with pipline , kubernetes and ovirt metadata fields. The _source and _all fields should be disabled for them as well. For testing, You should not see any metrics related data in the discovery tab. Only an option to create graphs in the visualize tab and dashboards. See attachement with the fields that are still have _source field. Can we raise priority on this? This one is blocking verification process and QA automation regarding integration of oVirt and Viaq and is required by RHV-QA. (In reply to Lukas Svaty from comment #4) > Can we raise priority on this? > > This one is blocking verification process and QA automation regarding > integration of oVirt and Viaq and is required by RHV-QA. What is it blocking? see BZ#1494190 I don't think it is a good way to remove this record if it will cause data not being populated in Discovery tab. If it's a requirement from the elasticsearch side, we should find another way to query the data in Discovery tab. @lvlcek - do you know of a good query, using curl and the query api, that can be used to determine if the collectd stats are being stored in Elasticsearch? (In reply to Lukas Svaty from comment #6) > see BZ#1494190 I don't think it is a good way to remove this record if it > will cause data not being populated in Discovery tab. If it's a requirement > from the elasticsearch side, we should find another way to query the data in > Discovery tab. Same as in other metrics solutions, like prometheus and graphite, the metrics samples are not required to be viewed per sample. We should find a solution for qa testing metrics. But, we should not save the source and all fields. Probably the best way would be pre-defined dashboards. Commits pushed to master at https://github.com/openshift/origin-aggregated-logging https://github.com/openshift/origin-aggregated-logging/commit/8bf4db038ca9d0a39b499a760ea1891535ab7e84 Bug 1494912 - metrics indexes should not save the _source https://bugzilla.redhat.com/show_bug.cgi?id=1494912 Completely disable the _source and _all for ovirt-metrics-* https://github.com/openshift/origin-aggregated-logging/commit/ef1639722a466254077161d5b22923fc6be102b8 Merge pull request #780 from richm/viaq-0.0.12 Automatic merge from submit-queue. Bug 1494912 - metrics indexes should not save the _source https://bugzilla.redhat.com/show_bug.cgi?id=1494912 Completely disable the _source and _all for ovirt-metrics-* /test @Shirly, OCP QE haven no experience to setup oVirt Env. Is there an oVirt Env for sharing? I think RHV QE should handle this bug. Hi Lukas, Could you help check if this bug could be verified? Thanks! verified in openshift-ansible-3.6.173.0.75-1.git.0.0a44128.el7.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3464 |
Created attachment 1330072 [details] kibana_screenshot Description of problem: As elastic recommend we should remove all _source for metrics index. Currently only fields under collectd namespace are removed. Version-Release number of selected component (if applicable): 3.6.1 How reproducible: 100% Steps to Reproduce: 1.Send metrics data from oVirt to the openshift setup. 2.Check kibana discovery tag 3. Actual results: We see the records that are collected Expected results: No dana should appear under for metrics. Additional info: