Bug 1436087 - [RFE] Update the postgresql collectd plugin queries
Summary: [RFE] Update the postgresql collectd plugin queries
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine-metrics
Classification: oVirt
Component: RFEs
Version: 1.0.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.1.3
: ---
Assignee: Shirly Radco
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: oVirt-Metrics-and-Logs 1456459
TreeView+ depends on / blocked
 
Reported: 2017-03-27 06:40 UTC by Shirly Radco
Modified: 2019-04-28 13:18 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-07-06 14:02:02 UTC
oVirt Team: Metrics
Embargoed:
rule-engine: ovirt-4.1+
rule-engine: exception+
ylavi: planning_ack+
rule-engine: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 74650 0 master MERGED collectd:Update the postgresql plugin queries 2020-05-04 12:05:55 UTC

Description Shirly Radco 2017-03-27 06:40:01 UTC
Description of problem:
We would like to collect the data that will help our users to gain an insight to the postgresql performance and be able to act upon them.

We want to include the following queries:
table_states, disk_io, disk_usage, custom_deadlocks

Comment 1 Lukas Svaty 2017-05-10 12:15:13 UTC
dont see any postgres stats under query:
collectd.plugin: postgresql

tested in ovirt-engine-metrics-1.0.3-1.el7ev.noarch

this is the content of template updated in patch, there is no postgresql specification thus I believe these patches did not get through

[root@ls-engine1 /]# cat /usr/share/ovirt-engine-metrics/setup/ansible/roles/ovirt_collectd/ovirt_engine/templates/builtins-conf-for-ovirt-engine.conf.j2
# The following are added to retrieve additional host info
LoadPlugin disk
#Disk plugin with defaults only

LoadPlugin cpu

<Plugin cpu>
        ReportByState true
        ReportByCpu true
        ValuesPercentage true
</Plugin>

LoadPlugin memory

<Plugin memory>
        ValuesAbsolute true
        ValuesPercentage true
</Plugin>

LoadPlugin load
#Load plugin with defaults only

LoadPlugin nfs
#NFS plugin no config values

LoadPlugin entropy
LoadPlugin swap

LoadPlugin df

<Plugin df>
       # ignore rootfs; else, the root file-system would appear twice, causing
       # one of the updates to fail and spam the log
        FSType rootfs
       # ignore the usual virtual / temporary file-systems
        FSType sysfs
        FSType proc
        FSType devtmpfs
        FSType devpts
        FSType tmpfs
        FSType fusectl
        FSType cgroup
        IgnoreSelected true
        ValuesAbsolute true
        ValuesPercentage true
</Plugin>

LoadPlugin interface

LoadPlugin aggregation

<Plugin aggregation>
   <Aggregation>
     Plugin "cpu"
     Type "percent"

     GroupBy "Host"
     GroupBy "TypeInstance"

     CalculateSum true
     CalculateAverage true
   </Aggregation>
</Plugin>

LoadPlugin processes

<Plugin processes>
     Process "fluentd"
     Process "collectd"
     ProcessMatch "ovirt-engine" "ovirt-engine\.xml"
</Plugin>


moving back to post as this is not available in 4.1.2

Comment 2 Lukas Svaty 2017-06-21 13:18:00 UTC
Can you specify which stats do we expect here?

At the moment only these values are reported:

these value types:
"type": "pg_db_size"
"type": "pg_blks", "
"type": "pg_n_tup_g"
"type": "pg_xact"

last two with different type_instances

Example sample:
{
  "_index": "ovirt-metrics-2017.06.21",
  "_type": "fluentd",
  "_id": "AVzKmJd-GNi9AvMt77i4",
  "_score": null,
  "_source": {
    "hostname": "ls-example.com",
    "collectd": {
      "dstypes": [
        "derive"
      ],
      "interval": 10,
      "plugin": "postgresql",
      "plugin_instance": "engine",
      "type": "pg_blks",
      "type_instance": "heap_hit",
      "postgresql": {
        "pg_blks": 2.69932371916975
      }
    },
    "tag": "project.ovirt-metrics-engine",
    "ovirt": {
      "entity": "engine"
    },
    "ipaddr4": "10.34.63.61",
    "ipaddr6": "2620:52:0:223c:21a:4aff:fe80:3f6c",
    "@timestamp": "2017-06-21T14:20:03+02:00"
  },
  "fields": {
    "@timestamp": [
      1498047603000
    ]
  },
  "highlight": {
    "collectd.plugin": [
      "@kibana-highlighted-field@postgresql@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1498047603000
  ]
}

Also this one should be either ONQA or moved to 4.1.4.
Do we consider storing information in tables (ovirt data)?

Comment 3 Shirly Radco 2017-06-22 07:22:55 UTC
Yes. These are the expected values.
I'll add documentation on all collcted metrics.
Postgres metrics are based on https://github.com/astro/collectd/blob/master/src/postgresql_default.conf

Comment 4 Lukas Svaty 2017-06-22 09:39:50 UTC
thus moving to verified in ovirt-engine-metrics-1.0.4.2-1.el7ev.noarch


Note You need to log in before you can comment on or make changes to this bug.