Bug 1458700 - [RFE] Add postgres to the collectd processes plugin configuration
Summary: [RFE] Add postgres to the collectd processes plugin configuration
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine-metrics
Classification: oVirt
Component: Generic
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.2.0
: ---
Assignee: Shirly Radco
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: oVirt-Metrics-and-Logs 1475135 1511549
TreeView+ depends on / blocked
 
Reported: 2017-06-05 09:21 UTC by Roy Golan
Modified: 2017-12-20 10:42 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-20 10:42:17 UTC
oVirt Team: Metrics
Embargoed:
sradco: ovirt-4.2?
lsvaty: testing_plan_complete+
rule-engine: planning_ack?
rule-engine: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 79472 0 'None' 'MERGED' 'collectd: collect postgresql to processes plugin' 2019-11-19 17:10:52 UTC

Description Roy Golan 2017-06-05 09:21:01 UTC
Description of problem:
'postgres' process metrics are needed, please add is to the Processes plugin confdiguration for the engine machine.

Expected results:
postgres process metrics is collected

Additional info:
Might this be redundant because of the postgres plugin?

Comment 1 Shirly Radco 2017-06-07 10:37:47 UTC
Should we see the results for all postgresql threads together or do we want to monitor specific threads?

Example threads:
postgres: logger process   
postgres: checkpointer process   
postgres: writer process   
postgres: wal writer process   
postgres: autovacuum launcher process   
postgres: stats collector process   
postgres: engine engine ::1(45418) idle
postgres: engine_history ovirt_engine_history 127.0.0.1(44808) idle
postgres: autovacuum worker process   ovirt_engine_history

Comment 2 Yaniv Lavi 2017-06-13 08:15:40 UTC
(In reply to Shirly Radco from comment #1)
> Should we see the results for all postgresql threads together or do we want
> to monitor specific threads?
> 
> Example threads:
> postgres: logger process   
> postgres: checkpointer process   
> postgres: writer process   
> postgres: wal writer process   
> postgres: autovacuum launcher process   
> postgres: stats collector process   
> postgres: engine engine ::1(45418) idle
> postgres: engine_history ovirt_engine_history 127.0.0.1(44808) idle
> postgres: autovacuum worker process   ovirt_engine_history

I would do both collect per thread and use aggregation plugin for the group.

Comment 3 Yaniv Kaul 2017-06-19 11:45:20 UTC
(In reply to Yaniv Lavi from comment #2)
> (In reply to Shirly Radco from comment #1)
> > Should we see the results for all postgresql threads together or do we want
> > to monitor specific threads?
> > 
> > Example threads:
> > postgres: logger process   
> > postgres: checkpointer process   
> > postgres: writer process   
> > postgres: wal writer process   
> > postgres: autovacuum launcher process   
> > postgres: stats collector process   
> > postgres: engine engine ::1(45418) idle
> > postgres: engine_history ovirt_engine_history 127.0.0.1(44808) idle

If possible, separate between engine PG and history PG.

> > postgres: autovacuum worker process   ovirt_engine_history
> 
> I would do both collect per thread and use aggregation plugin for the group.

Per process, not thread. And I agree about aggregation.

Comment 4 Roy Golan 2017-06-25 12:37:07 UTC
BTW, how do we handle remote db installations in ovirt-engine-metrics? in case missing this should be included as well.

Comment 5 Shirly Radco 2017-06-25 17:52:11 UTC
(In reply to Roy Golan from comment #4)
> BTW, how do we handle remote db installations in ovirt-engine-metrics? in
> case missing this should be included as well.

I don't see how this can be reported in a remote use case.
If process runs on a separate machine there is no way to collect it with the process plugin.

Comment 6 Roy Golan 2017-07-13 10:05:58 UTC
(In reply to Shirly Radco from comment #5)
> (In reply to Roy Golan from comment #4)
> > BTW, how do we handle remote db installations in ovirt-engine-metrics? in
> > case missing this should be included as well.
> 
> I don't see how this can be reported in a remote use case.
> If process runs on a separate machine there is no way to collect it with the
> process plugin.

By 'included' I mean to install collectd, fluentd on it as well - but I'm still not sure we _should_ do it because it is a less common setup, and in fact even not recommended.

Comment 7 Lukas Svaty 2017-10-04 11:38:59 UTC
I would disagree with Roy here that it is less common setup. For bigger scale environment I would prefer remote DB. Thus this BZ should consider running collectd and fluentd on the remote machine as well.

We already require ovirt repositories for pg95, thus I see no reason to have this.

Comment 8 Yaniv Kaul 2017-10-15 10:04:40 UTC
(In reply to Lukas Svaty from comment #7)
> I would disagree with Roy here that it is less common setup. For bigger
> scale environment I would prefer remote DB. Thus this BZ should consider
> running collectd and fluentd on the remote machine as well.

Why would you prefer a remote DB? Do you have evidence it improves the performance or resiliency of the setup?

Also, note that we do not monitor remote DB (that we did not set up).

Comment 9 Lukas Svaty 2017-10-16 07:32:07 UTC
Separation of data from the process (oVirt). In case I'm running more products requiring database I would want the data from databases to be grouped, preferably with redundancy, high availability and on a machine where admins do not need to connect all the time.

However, if we do not monitor yet *any* remote DB, I agree there is no need for starting now.

Comment 10 Shirly Radco 2017-11-09 15:44:24 UTC
This RFE adds postgres statistics: 
pg_writer, pg_wal_writer, pg_checkpointer_process, pg_autovacuum, pg_stats_collector, pg_engine, pg_ovirt_engine_history.

pg_engine and pg_ovirt_engine_history postgres processes do not support non default db names.

If db name are different then "engine" and "ovirt_engine_history"
metrics will be empty for the ovirt-engine and ovirt-engine-dwh in the processes plugin.

Added bug #1511549 to handle non default db names.

Comment 11 Lukas Svaty 2017-12-01 17:01:37 UTC
verified in ovirt-engine-metrics-1.1.1-0.3.beta2.20171114114644.el7ev.noarch

Comment 12 Sandro Bonazzola 2017-12-20 10:42:17 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.