Bug 1434575

Summary: [RFE] Update engine and hosts fluentd configurations to use rubygem-fluent-plugin-collectd-nest
Product: [oVirt] ovirt-engine-metrics Reporter: Shirly Radco <sradco>
Component: RFEsAssignee: Shirly Radco <sradco>
Status: CLOSED CURRENTRELEASE QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: bugs, lsvaty, pstehlik, sradco, stirabos, ylavi
Target Milestone: ovirt-4.1.3Keywords: FutureFeature
Target Release: 1.0.4.1Flags: rule-engine: ovirt-4.1+
ylavi: planning_ack+
rule-engine: devel_ack+
lsvaty: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-06 13:12:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Metrics RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1426897, 1426901, 1426903, 1426904    
Bug Blocks: 1402901, 1456459    

Description Shirly Radco 2017-03-21 19:49:48 UTC 
Description of problem:
The  rubygem-fluent-plugin-collectd-nest is required to formats Fluentd records in the proper ViaQ data model. We need to update the engine and hosts fluentd confs file to use it.
Comment 1 Lukas Svaty 2017-03-30 12:17:41 UTC 
Please provide verification steps for this one.
Comment 2 Shirly Radco 2017-06-20 13:27:51 UTC 
Steps to reproduce:

1. Set up metrics on the engine and hosts
2. Check the records in Kibana - 
     The plugin converts collectd fields to be nested under "collectd".
     The numeric value of the record is nested under "collectd"-> "plugin" -> "type". 
     A record can have one or more numeric values.

Previously this was done not in a dedicated plugin but in the config file.

Examples of expected results:

"collectd": {
      "interval": 10,
      "plugin": "processes",
      "type": "ps_state",
      "type_instance": "sleeping",
      "processes": {
        "ps_state": 272
      }
    }, 

"collectd": {
      "interval": 10,
      "plugin": "disk",
      "plugin_instance": "sda",
      "type": "disk_octets",
      "disk": {
        "disk_octets": {
          "read": 0,
          "write": 1899692.00845666
        }
      }
Comment 3 Lukas Svaty 2017-06-20 14:50:43 UTC 
(In reply to Shirly Radco from comment #2)
> Steps to reproduce:
> 
> 1. Set up metrics on the engine and hosts
> 2. Check the records in Kibana - 
>      The plugin converts collectd fields to be nested under "collectd".
>      The numeric value of the record is nested under "collectd"-> "plugin"
> -> "type". 
You sure about this?
>      A record can have one or more numeric values.
> 
> Previously this was done not in a dedicated plugin but in the config file.
> 
> Examples of expected results:
> 
> "collectd": {
>       "interval": 10,
>       "plugin": "processes",
>       "type": "ps_state",
>       "type_instance": "sleeping",
>       "processes": {
>         "ps_state": 272
IIUC this is under collectd->processes->ps_state and not collectd->plugin->type
>       }
>     }, 
> 
> "collectd": {
>       "interval": 10,
>       "plugin": "disk",
>       "plugin_instance": "sda",
>       "type": "disk_octets",
>       "disk": {
>         "disk_octets": {
>           "read": 0,
>           "write": 1899692.00845666

IIUC this is under collectd->disk->disk_octets->write and not collectd->plugin->type
>         }
>       }

Thus what am I really looking for, the data are indeed nested under collectd however does not fit your specification

example output on my machine:

{
  "_index": "ovirt-metrics-2017.06.20",
  "_type": "fluentd",
  "_id": "AVzEtQeqCiT5_6MSsdhE",
  "_score": 1,
  "_source": {
    "hostname": "******************************",
    "collectd": {
      "dstypes": [
        "gauge"
      ],
      "interval": 10,
      "plugin": "memory",
      "type": "memory",
      "type_instance": "buffered",
      "memory": {
        "memory": 2195456
      }
    },
    "tag": "project.ovirt-metrics-engine",
    "ovirt": {
      "entity": "engine"
    },
    "ipaddr4": "***********",
    "ipaddr6": "*****************************",
    "@timestamp": "2017-06-20T10:35:49+02:00"
  }
}

Is this correct they are nested under collectd?
What did you mean by 'The numeric value of the record is nested under "collectd"-> "plugin" -> "type".'?
Comment 4 Shirly Radco 2017-06-21 05:46:36 UTC 
This is ok.
This means the field name is built as 

"collectd" 
    { "memory"(plugin_field) 
        { "memory" (type_field) :2195456
        }
    }
Comment 5 Lukas Svaty 2017-06-21 07:27:02 UTC 
I see, thank you, due to comment #3 and #4 moving to verified