Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1410044

Summary:	[RFE] Configure collectd and fluentd on engine
Product:	[oVirt] ovirt-engine-metrics	Reporter:	Shirly Radco <sradco>
Component:	RFEs	Assignee:	Yedidyah Bar David <didi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Lukas Svaty <lsvaty>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	1.0.0	CC:	bugs, didi, lsvaty, sradco, ykaul, ylavi
Target Milestone:	ovirt-4.1.1-1	Keywords:	FutureFeature
Target Release:	---	Flags:	ykaul: needinfo+ rule-engine: ovirt-4.1+ rule-engine: exception+ gklein: testing_plan_complete- ylavi: planning_ack+ rule-engine: devel_ack+ lsvaty: testing_ack+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	1405813	Environment:
Last Closed:	2017-04-21 09:46:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1405809, 1405813
Bug Blocks:	1402901, 1434570

Description Shirly Radco 2017-01-04 10:30:20 UTC

Description of problem:

Once a user sets up a metrics store (bug 1405812), and configures the engine to use them (bug 1405809), a cron job should automatically configure engine to send metrics to this store. Current plan is to do this using ansible by enhancing:

https://github.com/centos-opstools/opstools-ansible

Comment 1 Yedidyah Bar David 2017-02-02 14:12:48 UTC

Please provide configuration files for the engine machine.

Can't see in [1] anything for postgresql.

I guess we also do not need statsd conf, and if we want to have a common subset for both engine and hosts, also perhaps move most of builtins-conf-for-ovirt.conf.j2 to a common file and keep there only host-specific stuff.

Similarly for fluentd ovirt_host/templates/ovirt-processing.conf.

[1] github.com/sradco/ovirt-metrics-conf

Comment 2 Shirly Radco 2017-02-13 11:24:27 UTC

(In reply to Yedidyah Bar David from comment #1)
> Please provide configuration files for the engine machine.
> 
> Can't see in [1] anything for postgresql.
> 
> I guess we also do not need statsd conf, and if we want to have a common

Also virt plugin is not relevant.

> subset for both engine and hosts, also perhaps move most of
> builtins-conf-for-ovirt.conf.j2 to a common file and keep there only
> host-specific stuff.


It is possible to have to have a common file and have separate host and engine specific file. But need to keep in mind that in Fluentd the order is important.

I think we should keep it simple for now and later improve it.

> 
> Similarly for fluentd ovirt_host/templates/ovirt-processing.conf.
> 
> [1] github.com/sradco/ovirt-metrics-conf

The engine config files are at 
https://github.com/sradco/ovirt-metrics-conf/blob/master/engine/

They do not include postgresql plugin yet. But you can use them.

Comment 3 Lukas Svaty 2017-03-30 08:46:10 UTC

as discussed in private thread, this need automatic population of config for postgres plugin, waiting for new patch, moving back to ASSIGNED

Comment 4 Red Hat Bugzilla Rules Engine 2017-03-30 08:46:17 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 5 Lukas Svaty 2017-04-04 14:33:34 UTC

removing 4.1.1 dep

Comment 6 Lukas Svaty 2017-04-07 11:53:08 UTC

[root@ls-engine1 ~]# service collectd status
Redirecting to /bin/systemctl status  collectd.service
● collectd.service - Collectd statistics daemon
   Loaded: loaded (/usr/lib/systemd/system/collectd.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2017-04-07 13:23:38 CEST; 12min ago
     Docs: man:collectd(1)
           man:collectd.conf(5)
 Main PID: 2514 (collectd)
   CGroup: /system.slice/collectd.service
           └─2514 /usr/sbin/collectd -C /etc/collectd.conf

[root@ls-engine1 ~]# service fluentd status
Redirecting to /bin/systemctl status  fluentd.service
● fluentd.service - Fluentd
   Loaded: loaded (/usr/lib/systemd/system/fluentd.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2017-04-07 13:23:17 CEST; 13min ago
     Docs: http://www.fluentd.org/
 Main PID: 2052 (fluentd)
   CGroup: /system.slice/fluentd.service
           ├─2052 /usr/bin/ruby /usr/bin/fluentd -c /etc/fluentd/fluent.conf
           └─2198 /usr/bin/ruby /usr/bin/fluentd -c /etc/fluentd/fluent.conf

values are being reported to kibana under ovirt.entity: engine

verified in ovirt-engine-metrics-1.0.2-1.el7ev.noarch

Yaniv:
as for  
"Yaniv Kaul 2017-04-02 03:45:02 EDT
Target Milestone: ovirt-4.1.1-1 → ovirt-4.1.2"

as this was verified  4.1.1-1 can we re-target to 4.1.1-1 thus comment#5 will be obsolete and we can re-add and track it there



rest of the engine bugs are tracked here - BZ#1429861 and BZ#1429875 are scheduled for 4.1.2