Bug 1595375

Summary: foreman-maintain does not rotate its log file
Product: Red Hat Satellite Reporter: Christian Marineau <cmarinea>
Component: Satellite MaintainAssignee: Anurag Patel <apatel>
Status: CLOSED NOTABUG QA Contact: Katello QA List <katello-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.0CC: apatel, inecas, kgaikwad, mbacovsk
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 06:57:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Christian Marineau 2018-06-26 19:04:13 UTC
Description of problem:
foreman-maintain keeps appending the same file /var/log/foreman-maintain/foreman-maintain.log every time it runs.

Version-Release number of selected component (if applicable):
6.3

How reproducible:
100%

Steps to Reproduce:
1. Run foreman-maintain multiple time
2. Verify the logs 
   # ls -lrt /var/log/foreman-maintain/


Actual results:
Only one file is present:
  /var/log/foreman-maintain/foreman-maintain.log

Expected results:
  Have a rotation at each run like satellite-installer does, which could look like this:
    /var/log/foreman-maintain/foreman-maintain.log
    /var/log/foreman-maintain/foreman-maintain.1.log
    /var/log/foreman-maintain/foreman-maintain.2.log

Additional info:
  While this has no technical impact on the functionality of the tool itself, it makes troubleshooting more fastidious. 

  When trying to determine when and why foreman-maintain encountered an error, we have to figure out the date which could be on different line.

  As an example, foreman-maintain could have failed on day 1 with error A, then succeed, then failed on day 3 with error B, then failed on day 4 with error C. Now we try to identify when it succeed and when/why it lastly failed to determine the proper fix, but the current log format adds unnecessary complexity.

Comment 1 Kavita 2018-06-27 06:57:01 UTC
Hi Christian,

Foreman maintain does log rotate.

Default file size set to 10000KB. 

Once file size reach to this limit, it will rotate the log file i.e creates backup log file for old logs - /var/log/foreman-maintain/foreman-maintain.log.0 

and continue logging into /var/log/foreman-maintain/foreman-maintain.log file.

You can modify this file size limit in /etc/foreman-maintain/foreman_maintain.yml using log_file_size option as per your requirement.

I would suggest you to try above configuration option & check log rotate. If you find any issue in this please feel free to reopen this ticket.

Comment 2 Christian Marineau 2018-06-27 15:36:23 UTC
(In reply to Kavita from comment #1)
> Hi Christian,
> 
> Foreman maintain does log rotate.
> 
> Default file size set to 10000KB. 
> 
> Once file size reach to this limit, it will rotate the log file i.e creates
> backup log file for old logs -
> /var/log/foreman-maintain/foreman-maintain.log.0 
> 
> and continue logging into /var/log/foreman-maintain/foreman-maintain.log
> file.
> 
> You can modify this file size limit in
> /etc/foreman-maintain/foreman_maintain.yml using log_file_size option as per
> your requirement.
> 
> I would suggest you to try above configuration option & check log rotate. If
> you find any issue in this please feel free to reopen this ticket.

Hi,

Thanks for your comment. Actually I was not accurate enough. What I wanted to highlight is the fact that the current log format is creating an unnecessary complexity. We are having case where we need to understand when and why foreman-maintain fails in the past. The same task done in the satellite-installer log is way easier since a new log file is create for every run. Then we can quickly know which run had an error and which ones were successful.

Was there a reason why we did not use the same kind of rotation for foreman-maintain? And would it be possible to verify the feasibility of this?

I also think that the BZ title may not be accurate, but the need is well explained in the description and the goal is to help support people to better identify future improvement of foreman-maintain.

Thanks.

Comment 3 Anurag Patel 2018-06-28 09:51:55 UTC
Hello Christian,

satellite-installer does have a rolling_file logger which is useful for that tool.

For foreman-maintain, however, we decided to rotate on filesize because satellite upgrade's heavy lifting is done by satellite-installer itself (invoked internally by f-m) and it already has a familiar logging structure in place.

In any case, we'll bring this topic for discussion internally and update here.

Anurag

Comment 4 Christian Marineau 2018-06-28 13:19:54 UTC
(In reply to Anurag Patel from comment #3)
> Hello Christian,
> 
> satellite-installer does have a rolling_file logger which is useful for that
> tool.
> 
> For foreman-maintain, however, we decided to rotate on filesize because
> satellite upgrade's heavy lifting is done by satellite-installer itself
> (invoked internally by f-m) and it already has a familiar logging structure
> in place.
> 
> In any case, we'll bring this topic for discussion internally and update
> here.
> 
> Anurag

Hi,

Thanks for your reply. Actually there are some additional steps like "yum update" and self-healing steps for known issue that can't be tracked via the satellite-installer. 

To give you a real-life example, please look at this KCS https://access.redhat.com/solutions/3497291

There are a lot of possible causes for the situation explained in the KCS and we would need to have a more efficient way to understand what is going on with foreman-maintain. Actually we basically need to search the entire file and figure out when the run was success and when it fails, and the current format is not helpful for support teams.

Thanks!

-Christian