Bug 1020790

we'll cover this via bug 971853 - sos will add "last log" and "all logs" at this point, and we are dependent on sos for this.

*** This bug has been marked as a duplicate of bug 971853 ***

Please see https://bugzilla.redhat.com/show_bug.cgi?id=1304394#c4 for details.
We will provide option to collect based on size.

Sandro, do you feel a simple 'collect only last logs per each component' is a good enough solution for this one? Do not collect rolled-over logs, for example? 
I think for 4.0 it's fairly simple to implement and should be good enough. Thoughts?

(In reply to Yaniv Kaul from comment #8)
> Sandro, do you feel a simple 'collect only last logs per each component' is
> a good enough solution for this one? Do not collect rolled-over logs, for
> example? 
> I think for 4.0 it's fairly simple to implement and should be good enough.
> Thoughts?

log-collector relies on sos reports collection so if sos support "Do not collect rolled-over logs" we can use it.
As far as I can tell this option doesn't exist.
What has been suggested is to expose --log-size=LOG_SIZE   (set a limit on the size of collected logs) to be passed to underlying sos (meaning that each host will collect max LOG_SIZE logs).

We can combine it with plugin option:
 logs.log_days 1 (the number of days logs to collect) for logs handled by logs plugin.
in order to limit some of the logs to last 24 hours.

(In reply to Sandro Bonazzola from comment #9)
> (In reply to Yaniv Kaul from comment #8)
> > Sandro, do you feel a simple 'collect only last logs per each component' is
> > a good enough solution for this one? Do not collect rolled-over logs, for
> > example? 
> > I think for 4.0 it's fairly simple to implement and should be good enough.
> > Thoughts?
> 
> log-collector relies on sos reports collection so if sos support "Do not
> collect rolled-over logs" we can use it.
> As far as I can tell this option doesn't exist.
> What has been suggested is to expose --log-size=LOG_SIZE   (set a limit on
> the size of collected logs) to be passed to underlying sos (meaning that
> each host will collect max LOG_SIZE logs).

I don't know how that works - does it mean it may end up collecting only vdsm logs, and skipping supervdsm, for example?

> 
> We can combine it with plugin option:
>  logs.log_days 1 (the number of days logs to collect) for logs handled by
> logs plugin.
> in order to limit some of the logs to last 24 hours.

This looks like the right approach to me.

Verifying based Doc text as other info were not posted to this bug:

tested on: ovirt-log-collector-4.2.0-1.el7ev.noarch

"The ovirt-log-collector can now limit the maximum size of logs collected in order to capture relevant logs only. The size of the logs may be extremely large in environments with large numbers of hosts or a large number of exceptions, so ovirt-log-collector can now arbitrarily limit the size of logs collected, defaulting to the last day only."

"defaulting to the last day only.":
- Checking host logs, /var/log/messages contains more than 1 day of logs
- Checking engine logs, /var/log/ovirt-engine/engine.log contains more than 1 day of logs

Trying to see param for it:
# ovirt-log-collector --help | grep size
  --log-size=SIZE       maximum log size to collect on each host in MiB

Trying out the param compared to default run:
# ovirt-log-collector --log-size=1
...
log-collector-data]# du -skh /tmp/sosreport-LogCollector-20180126164815/log-collector-data/host-*
25M	host-1
25M	host-2
31M	host-3


# ovirt-log-collector --log-size=100
...
log-collector-data]# du -skh /tmp/sosreport-LogCollector-20180126164815/log-collector-data/host-*
25M	host-1
25M	host-2
31M	host-3


# ovirt-log-collector
...
log-collector-data]# du -skh /tmp/sosreport-LogCollector-20180126164815/log-collector-data/host-*
25M	host-1
25M	host-2
31M	host-3

limiting log-size does not seem to have any effect.

Moving to ASSSIGNED for now, and adding needinfo on Ryan after the conversation with DEV. If there is anything I am not aware of please correct me and I'll reverify.

Douglas -

I won't have an environment where I can look at this until next week. Can you take a look?

Douglas, I'm reassigning this, as you're the maintainer.

The --log-size option is correctly passed to sosreport.

However, sosreport ignores this, because two conflicting sets of option are passed:

We are passing `-o logs,...` combined with `--all-logs` (or its equivalent for older versions). --all-logs ignores --log-size.

--all-logs is also unnecessary when combined with -o, which limits the set of options. I'm actually not sure what the behavior for sosreport is when these option are combined, so I'll leave it up to you.

Probably we want to remove --all-logs.

Moving to Ido since Douglas is on training this week

Closing this RFE. Please reopen if still needed.
We have made considerable improvement to not get to this state.

Adding warning, limiting to one host per cluster and so on.

BZ<2>Jira Resync