Bug 1405813 - [RFE] Configure collectd and fluentd on hosts
Summary: [RFE] Configure collectd and fluentd on hosts
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: RFEs
Version: ---
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.1.0-rc
: 4.1.0.3
Assignee: Yedidyah Bar David
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On: 1392784 1405809 1414277 1418985
Blocks: oVirt-Metrics-and-Logs 1410044 1451625
TreeView+ depends on / blocked
 
Reported: 2016-12-18 15:42 UTC by Yedidyah Bar David
Modified: 2019-04-28 13:59 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
A script is now supplied to configure collectd and fluentd on hosts to send statistics to a central store.
Clone Of:
: 1410044 (view as bug list)
Environment:
Last Closed: 2017-04-27 09:36:23 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.1+
ykaul: exception+
gklein: testing_plan_complete-
ylavi: planning_ack+
sbonazzo: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 69908 0 master MERGED packaging: ovirt-engine-hosts-ansible-inventory: Add up_metrics hosts 2020-06-17 14:20:15 UTC
oVirt gerrit 69927 0 master MERGED packaging: ovirt-engine-hosts-ansible-inventory: Init groups 2020-06-17 14:20:15 UTC
oVirt gerrit 70624 0 ovirt-engine-4.1 MERGED packaging: ovirt-engine-hosts-ansible-inventory: Add up_metrics hosts 2020-06-17 14:20:15 UTC
oVirt gerrit 70625 0 ovirt-engine-4.1 MERGED packaging: ovirt-engine-hosts-ansible-inventory: Init groups 2020-06-17 14:20:15 UTC
oVirt gerrit 70672 0 master MERGED packaging: Configure hosts for metrics 2020-06-17 14:20:15 UTC
oVirt gerrit 70708 0 ovirt-engine-4.1 MERGED packaging: Configure hosts for metrics 2020-06-17 14:20:15 UTC
oVirt gerrit 70722 0 master MERGED packaging: Configure hosts for metrics: Fixes 2020-06-17 14:20:15 UTC
oVirt gerrit 70792 0 ovirt-engine-4.1 MERGED packaging: Configure hosts for metrics: Fixes 2020-06-17 14:20:15 UTC
oVirt gerrit 70794 0 master ABANDONED spec: Require ovirt-host-deploy 1.6.1 2020-06-17 14:20:15 UTC
oVirt gerrit 70799 0 master MERGED packaging: Configure hosts for metrics: cleanups 2020-06-17 14:20:15 UTC
oVirt gerrit 70812 0 ovirt-engine-4.1 MERGED packaging: Configure hosts for metrics: cleanups 2020-06-17 14:20:15 UTC

Description Yedidyah Bar David 2016-12-18 15:42:55 UTC
Description of problem:

Once a user sets up a metrics store (bug 1405812), and configures the engine to use them (bug 1405809), a cron job should automatically configure all active hosts with compatibility level 4.1 to send metrics to this store. Current plan is to do this using ansible by enhancing:

https://github.com/centos-opstools/opstools-ansible

Comment 1 Yedidyah Bar David 2017-01-23 14:50:10 UTC
Note for QE/Doc. To test metrics:

1. Install and configure fluentd, elasticsearch and kibana on some machine. Currently it should be a new machine, separate from the engine and the hosts. This will be called "the metrics store machine". See bug 1405812 for more info. You can use [1] for now. Current plan is to eventually use OCP with common logging, not sure about docs for that. You can find example configuration files for fluentd in [2].

2. Install and setup an engine.

3. Install and activate one or more 4.1 hosts.

4. On the engine machine, create a directory /etc/ovirt-engine/metrics.

5. Copy the ca certificate used in the metrics store machine to /etc/ovirt-engine/metrics/fluentd_ca_cert.pem on the engine machine.

6. Create on the engine machine a file /etc/ovirt-engine/metrics/config.yml with the following two lines:
fluentd_fluentd_host: my.fluentd.host
fluentd_shared_key: my_shared_key
Replace "my.fluentd.host" with the fqdn of the metrics store machine, and "my_shared_key" with the shared key configured in fluentd on the metrics store machine.

7. Run on the engine machine, as root:
/usr/share/ovirt-engine/setup/ansible/configure_ovirt_hosts_for_metrics.sh

It runs ansible and should finish without errors.
When finished, you should be able to see statistics in the kibana console about your hosts, VMs, etc.

We should probably have more detailed documentation for this feature. Once written, it should appear in [3].

[1] https://sradcoblog.wordpress.com/2016/07/19/ovirt-metrics-elk/
[2] https://github.com/sradco/ovirt-metrics-conf/tree/master/metrics-store
[3] http://www.ovirt.org/develop/release-management/features/engine/metrics-store/

Comment 2 Yedidyah Bar David 2017-01-23 14:52:35 UTC
See previous comment for some documentation. Setting "require doc text" to "-" because I guess we need a proper doc bug for this feature.

Comment 3 Lukas Svaty 2017-02-03 11:10:11 UTC
two of the dependant bugs are not fixed yet, moving back to ASSIGNED

Comment 4 Red Hat Bugzilla Rules Engine 2017-02-03 11:10:17 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 5 Yedidyah Bar David 2017-02-06 15:38:20 UTC
Moving back to QE. One dependent bug closed, other moved to QE as well.

Comment 6 Yedidyah Bar David 2017-02-14 13:12:29 UTC
Setting needinfo on myself to move this bug to a new product once we create one, for https://gerrit.ovirt.org/#/admin/projects/ovirt-metrics .

Comment 7 Red Hat Bugzilla Rules Engine 2017-02-14 13:12:35 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 8 Yaniv Lavi 2017-02-15 11:52:29 UTC
Why is this back to assigned?

Comment 9 Yedidyah Bar David 2017-02-15 12:45:07 UTC
(In reply to Yaniv Dary from comment #8)
> Why is this back to assigned?

See comment 6. But I agree it does not make much sense. Moving back to QE. Created bug 1422505 for that.

Comment 10 Yedidyah Bar David 2017-03-02 08:19:49 UTC
An update:

We decided to split this off the engine.

Implications for developers:

There is now a new gerrit.ovirt.org repo called 'ovirt-engine-metrics'.

Implications for everyone, also updating comment 1:

There is a new package called ovirt-engine-metrics. It replaces the package ovirt-engine-hosts-ansible-inventory . On upgrade from 4.1.0, this should happen automatically.

Configuration for this is now in /etc/ovirt-engine-metrics .

It already provides there a file config.yml.example, which you can copy to config.yml and edit.

This file should also include the ca cert path, which is now not required to be copied there to a specific name.

You should then run:

/usr/share/ovirt-engine-metrics/setup/ansible/configure_ovirt_hosts_for_metrics.sh

Comment 11 Lukas Svaty 2017-03-29 12:55:33 UTC
verified with ovirt-engine-metrics-1.0.0-1.el7ev.noarch


Note You need to log in before you can comment on or make changes to this bug.