Bug 1475900 - [RFE] Collect the logs from vdsm.log to the metrics store
Summary: [RFE] Collect the logs from vdsm.log to the metrics store
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine-metrics
Classification: oVirt
Component: RFEs
Version: 1.0.5
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ovirt-4.1.8
: ---
Assignee: Shirly Radco
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: oVirt-Metrics-and-Logs 1475135 1513032 1513703
TreeView+ depends on / blocked
 
Reported: 2017-07-27 14:57 UTC by Shirly Radco
Modified: 2023-09-07 18:55 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-11 16:30:10 UTC
oVirt Team: Metrics
Embargoed:
rule-engine: ovirt-4.1+
rule-engine: ovirt-4.2+
lsvaty: testing_plan_complete+
ylavi: planning_ack+
rule-engine: devel_ack+
pstehlik: testing_ack+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 80010 0 'None' 'MERGED' 'fluentd: collect vdsm.log' 2019-11-28 14:23:53 UTC
oVirt gerrit 84011 0 'None' 'MERGED' 'Update collection to collect vdsm.log by default' 2019-11-28 14:23:53 UTC
oVirt gerrit 84162 0 'None' 'MERGED' 'fluentd: collect vdsm.log' 2019-11-28 14:23:53 UTC
oVirt gerrit 84163 0 'None' 'MERGED' 'Update collection to collect vdsm.log by default' 2019-11-28 14:23:53 UTC

Description Shirly Radco 2017-07-27 14:57:23 UTC 
Description of problem:
Collect the logs from vdsm.log to the metrics store
Comment 2 Shirly Radco 2017-10-16 09:43:09 UTC 
Please provide the use cases for using this log.
Comment 3 Yaniv Lavi 2017-10-29 17:04:21 UTC 
(In reply to Shirly Radco from comment #2)
> Please provide the use cases for using this log.

Debugging engine-host issues in a single view.
Exploring VMs and tasks bottlenecks on the hosts.
Comment 4 Shirly Radco 2017-11-09 12:58:41 UTC 
configure_host_logs_processing is set to false by default.

In order to test user can run
/usr/share/ovirt-engine-metrics/setup/ansible/configure_ovirt_machines_for_metrics.sh -e "configure_host_logs_processing=true"

or add 

configure_host_logs_processing: true

to the /etc/ovirt-engine-metrics/config.yml file.

When updating back to configure_host_logs_processing=false the configuration file is removed from /etc/fluentd/config.d/ and collectd is restarted.
Comment 5 Lukas Svaty 2017-11-09 13:04:35 UTC 
Hi Shirly, 

I do not agree with the decision of having it off by default, I see an inconsistency here between 2 collection utilities, log-collector and metrics store.

Log-collector is gathering logs from hosts by default while metrics store is not. 

Where did the decision to have it disabled by default come from? Can we change it to be enabled by default?

LS
Comment 6 Yaniv Lavi 2017-11-09 13:11:47 UTC 
Since we see that system resources is not affected, let's move to be on by default.
Comment 7 Shirly Radco 2017-11-13 12:36:19 UTC 
I have updated the default value to true.
vdsm.log will now be collected by default.
Comment 8 Shirly Radco 2017-11-15 10:49:12 UTC 
Yaniv L requested this to be backport also to 4.1.
Comment 9 Lukas Svaty 2017-11-29 11:31:08 UTC 
Currently index for logs is not being created in ViaQ setup, thus this is blocking this verification.

More over I do not see any parsing information of vdsmd.log so it seems to me like FailedQA. Shirly any advise on verification steps here?

For engine log:
Nov 29 11:55:44 example.com fluentd[15159]: 2017-11-29 11:55:44 +0100 [info]: following tail of /var/log/ovirt-engine/engine.log

Nothing for vdsmd.log
Comment 10 Lukas Svaty 2017-11-30 14:20:15 UTC 
ignore the last comment, ofcourse you cant follow both engine.log and vdsm.log on a single machine as they are on different machines (host vs engine)

verified in ovirt-engine-metrics-1.0.8-1.el7ev.noarch
Comment 11 Yaniv Lavi 2017-12-03 13:24:22 UTC 
*** Bug 1388611 has been marked as a duplicate of this bug. ***
Note You need to log in before you can comment on or make changes to this bug.