Bug 1424997

Summary: [RFE] Update fluentd configuration to fit the common data model
Product: [oVirt] ovirt-engine-metrics Reporter: Shirly Radco <sradco>
Component: RFEsAssignee: Shirly Radco <sradco>
Status: CLOSED CURRENTRELEASE QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: bugs, didi, juwu, pstehlik, sradco
Target Milestone: ovirt-4.1.1-1Keywords: CodeChange, FutureFeature
Target Release: 1.0.1Flags: sradco: ovirt-4.1?
sradco: ovirt-4.2?
sradco: exception?
lsvaty: testing_plan_complete-
sradco: planning_ack?
sradco: devel_ack+
pstehlik: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
With this update, the data sent to the remote metrics store is now in the common data model format.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-04-21 09:31:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Metrics RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1402901, 1428386    

Description Shirly Radco 2017-02-20 10:11:28 UTC
Description of problem:

As part of the common data model we need to update the fluentd.conf so data we be saved in the metrics store in the pre-defined common data model.

Comment 2 Lukas Svaty 2017-03-29 14:32:03 UTC
what needs to be tested here?

Comment 3 Lukas Svaty 2017-03-30 10:44:07 UTC
after discussion with DEV adding CodeChange and moving to verified without qa process

Comment 4 Yedidyah Bar David 2017-03-30 11:32:32 UTC
I am by no means a "data model" expert, but I think I disagree.

What if in the future the data model is changed, so all parties do their changes (client fluentd, elasticsearch, perhaps server fluentd, kibana (or customization of it))? And what happens if there is an agreement about such a change, but only some parts are changed? Won't the user see anything anywhere? Won't fluentd or elastic fail, or kibana will show wrong/different results? All of these are user visible.

IMO CodeChange is reserved for refactoring inside a single project that if done correctly is invisible to users.

Comment 5 Shirly Radco 2017-03-30 11:59:42 UTC
The data model is still wip. 
I'll add documentation and it should tested on https://bugzilla.redhat.com/1434575
It still has frequent updates like adding the engine_fqdn, env_name etc.

Comment 6 Lukas Svaty 2017-03-30 12:17:06 UTC
Dependant bug is ovirt-4.1.2 this one is planned for ovirt-4.1.1 please move it accordingly or remove the dependency