Description of problem: The rubygem-fluent-plugin-collectd-nest is required to formats Fluentd records in the proper ViaQ data model. We need to update the engine and hosts fluentd confs file to use it.
Please provide verification steps for this one.
Steps to reproduce: 1. Set up metrics on the engine and hosts 2. Check the records in Kibana - The plugin converts collectd fields to be nested under "collectd". The numeric value of the record is nested under "collectd"-> "plugin" -> "type". A record can have one or more numeric values. Previously this was done not in a dedicated plugin but in the config file. Examples of expected results: "collectd": { "interval": 10, "plugin": "processes", "type": "ps_state", "type_instance": "sleeping", "processes": { "ps_state": 272 } }, "collectd": { "interval": 10, "plugin": "disk", "plugin_instance": "sda", "type": "disk_octets", "disk": { "disk_octets": { "read": 0, "write": 1899692.00845666 } }
(In reply to Shirly Radco from comment #2) > Steps to reproduce: > > 1. Set up metrics on the engine and hosts > 2. Check the records in Kibana - > The plugin converts collectd fields to be nested under "collectd". > The numeric value of the record is nested under "collectd"-> "plugin" > -> "type". You sure about this? > A record can have one or more numeric values. > > Previously this was done not in a dedicated plugin but in the config file. > > Examples of expected results: > > "collectd": { > "interval": 10, > "plugin": "processes", > "type": "ps_state", > "type_instance": "sleeping", > "processes": { > "ps_state": 272 IIUC this is under collectd->processes->ps_state and not collectd->plugin->type > } > }, > > "collectd": { > "interval": 10, > "plugin": "disk", > "plugin_instance": "sda", > "type": "disk_octets", > "disk": { > "disk_octets": { > "read": 0, > "write": 1899692.00845666 IIUC this is under collectd->disk->disk_octets->write and not collectd->plugin->type > } > } Thus what am I really looking for, the data are indeed nested under collectd however does not fit your specification example output on my machine: { "_index": "ovirt-metrics-2017.06.20", "_type": "fluentd", "_id": "AVzEtQeqCiT5_6MSsdhE", "_score": 1, "_source": { "hostname": "******************************", "collectd": { "dstypes": [ "gauge" ], "interval": 10, "plugin": "memory", "type": "memory", "type_instance": "buffered", "memory": { "memory": 2195456 } }, "tag": "project.ovirt-metrics-engine", "ovirt": { "entity": "engine" }, "ipaddr4": "***********", "ipaddr6": "*****************************", "@timestamp": "2017-06-20T10:35:49+02:00" } } Is this correct they are nested under collectd? What did you mean by 'The numeric value of the record is nested under "collectd"-> "plugin" -> "type".'?
This is ok. This means the field name is built as "collectd" { "memory"(plugin_field) { "memory" (type_field) :2195456 } }
I see, thank you, due to comment #3 and #4 moving to verified