Bug 1570083

Summary: [Logs] Karaf logs are being rotated at 16M instead of 10G and maxrollover is set 7 leading to loss of log data
Product: Red Hat OpenStack Reporter: Sai Sindhur Malleni <smalleni>
Component: puppet-opendaylightAssignee: Janki <jchhatba>
Status: CLOSED NOTABUG QA Contact: Itzik Brown <itbrown>
Severity: medium Docs Contact:
Priority: medium    
Version: 13.0 (Queens)CC: aadam, jchhatba, jjoyce, jschluet, mkolesni, slinaber, tvignaud, vorburger
Target Milestone: z1Keywords: Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: Logs
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
N/A
Last Closed: 2018-06-14 15:06:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Sai Sindhur Malleni 2018-04-20 15:24:24 UTC
Description of problem: Karaf logs for ODL are being rotated every 16M. Earlier they were being rotated every 10G and looking puppet-opendaylight, it looks like the default is still 10G. Not sure if tripleo is missing it up to cause


Version-Release number of selected component (if applicable):
OSP 13
opendaylight-8.0.0-5.el7ost.noarch 
puppet-opendaylight-8.1.0-0.20180321182556.45c4db7.el7ost.noarch

How reproducible:
100 %

Steps to Reproduce:
1. Install ODL + OSP13 with defaults configuration
2.
3.

Actual results:
Log files are being rotated every 17M leading to 10s of karaf log files

Expected results:
Log files were previously being rotated every 10G

Additional info:
In etc/org.ops4j.pax.logging.cfg
log4j2.appender.rolling.policies.size.size = 16MB

Comment 1 Sai Sindhur Malleni 2018-04-20 15:29:15 UTC
Looks like puppet-opendaylight tries to configure 10G by default?
https://github.com/opendaylight/integration-packaging-puppet-opendaylight/blob/a6b1e01447b6e50b7b9b9250f106434b880c6e57/manifests/config.pp#L246

Comment 2 Michael Vorburger 2018-04-24 15:59:03 UTC
> External Bug ID: OpenDaylight Bug ODLPARENT-154

This issue does NOT have to await ODLPARENT-154 - it's (presumably) easy to do this locally in puppet-opendaylight - but it's really an indication that the default upstream could be better.

Comment 3 Sai Sindhur Malleni 2018-05-07 14:54:24 UTC
Hey Michael,

Was wondering about the status of this downstream? Also looks like we are capping the rotated logs at 7 files. This lead to me losing a ot of valuable logs when running longevity tests over the weekend. Can we increase the max rollovers atleast if we cant increase the log file size?

Comment 4 Janki 2018-05-09 10:18:48 UTC
Puppet configures rollover at 2 with each file size of 10GB. Puppet also has a logic that

if log_to_console:
    "logs to console + file"
else:
  configure file with above said size and rollover

and TripleO takes puppet to log to console. So the whole "else" section is not executed.

Comment 5 Sai Sindhur Malleni 2018-05-14 14:39:15 UTC
So who is setting the rollover to 7 in this case?

Comment 7 Janki 2018-05-16 12:22:33 UTC
I will edit the above explained if-else logic in puppet-odl to also configure file size and rollover even if log_mechanism mentioned in TripleO is "console".

Comment 8 Janki 2018-06-14 15:06:17 UTC
Since we are moving to logging to file in ODL, the logs files will be accordingly configured. This work is tracked at 

https://bugzilla.redhat.com/show_bug.cgi?id=1559055

No more changes needed for this bug. Hence closing it.