Bug 1265463 - log rotate not working on the appliance
log rotate not working on the appliance
Status: CLOSED ERRATA
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance (Show other bugs)
5.5.0
Unspecified Unspecified
unspecified Severity high
: GA
: 5.5.0
Assigned To: Nick Carboni
Alex Newman
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-22 21:37 EDT by Dave Johnson
Modified: 2015-12-08 08:32 EST (History)
5 users (show)

See Also:
Fixed In Version: 5.5.0.10
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-08 08:32:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dave Johnson 2015-09-22 21:37:03 EDT
Description of problem:
============================
Running a appliance for 4 days, no log rotate...

[root@host-192-168-55-149 log]# ll
total 277048
drwxr-xr-x. 2 root root        90 Sep 17 05:27 apache
-rw-r--r--. 1 root root        66 Sep 16 10:34 api.log
-rw-r--r--. 1 root root       590 Sep 17 05:26 appliance_console.log
-rw-r--r--. 1 root root     15499 Sep 22 21:14 audit.log
-rw-r--r--. 1 root root    188511 Sep 22 21:27 automation.log
-rw-r--r--. 1 root root      2062 Sep 22 20:19 aws.log
-rw-r--r--. 1 root root 119570860 Sep 22 21:31 evm.log
-rw-r--r--. 1 root root        66 Sep 16 10:34 fog.log
-rw-r--r--. 1 root root      3961 Sep 18 11:52 gem_list.txt
-rw-r--r--. 1 root root        66 Sep 16 10:34 kubernetes.log
-rw-r--r--. 1 root root     60584 Sep 18 11:52 last_startup.txt
-rw-r--r--. 1 root root    471550 Sep 22 21:30 miq_ntpdate.log
-rw-r--r--. 1 root root     18006 Sep 18 11:52 package_list_rpm.txt
-rw-r--r--. 1 root root    288745 Sep 22 21:28 policy.log
-rw-r--r--. 1 root root   1082210 Sep 22 21:29 production.log
-rw-r--r--. 1 root root   1867793 Sep 22 21:31 rhevm.log
-rw-r--r--. 1 root root        66 Sep 16 10:34 scvmm.log
-rw-r--r--. 1 root root 121190230 Sep 22 21:30 top_output.log
-rw-r--r--. 1 root root        66 Sep 16 10:34 vim.log
-rw-r--r--. 1 root root    527737 Sep 22 21:30 vmstat_output.log
[root@host-192-168-55-149 log]# uptime
 21:31:32 up 4 days,  9:48,  1 user,  load average: 0.42, 0.67, 0.95



Version-Release number of selected component (if applicable):
====================================
5.5.0.1


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 3 Alex Newman 2015-11-02 09:55:28 EST
Additional Details:

Log path: /var/www/miq/vmdb/log
Check mask: {*.log}-{yyyymmdd}.gz
Comment 4 Alex Newman 2015-11-02 09:59:54 EST
[root@s-appl-downstream-55z-151021-oyvllkg1 log]# ll
total 1955504
drwxr-xr-x. 2 root root         90 Oct 22 08:23 apache
-rw-r--r--. 1 root root         66 Oct 21 15:49 api.log
-rw-r--r--. 1 root root        594 Oct 22 08:20 appliance_console.log
-rw-r--r--. 1 root root       5055 Nov  2 09:18 audit.log
-rw-r--r--. 1 root root  226521044 Nov  2 09:54 automation.log
-rw-r--r--. 1 root root         66 Oct 21 15:49 aws.log
-rw-r--r--. 1 root root 1236876887 Nov  2 09:55 evm.log
-rw-r--r--. 1 root root         66 Oct 21 15:49 fog.log
-rw-r--r--. 1 root root       8754 Oct 26 09:07 gem_list.txt
-rw-r--r--. 1 root root         66 Oct 21 15:49 kubernetes.log
-rw-r--r--. 1 root root      60920 Oct 26 09:06 last_startup.txt
-rw-r--r--. 1 root root     755129 Nov  2 09:45 miq_ntpdate.log
-rw-r--r--. 1 root root         90 Oct 26 09:07 package_list_rpm.txt
-rw-r--r--. 1 root root      47033 Nov  2 09:08 policy.log
-rw-r--r--. 1 root root     568876 Nov  2 09:18 production.log
-rw-r--r--. 1 root root         66 Oct 21 15:49 rhevm.log
-rw-r--r--. 1 root root         66 Oct 21 15:49 scvmm.log
-rw-r--r--. 1 root root  238695700 Nov  2 09:55 top_output.log
-rw-r--r--. 1 root root         66 Oct 21 15:49 vim.log
-rw-r--r--. 1 root root     857278 Nov  2 09:55 vmstat_output.log
[root@s-appl-downstream-55z-151021-oyvllkg1 log]# uptime
 09:56:04 up 7 days,  1:52,  1 user,  load average: 0.13, 0.19, 0.23
Comment 5 Nick Carboni 2015-11-02 10:10:27 EST
Can you check /var/www/miq/vmdb/appliance_console.log for any messages relating to log rotation?

Such as:
"Rotating file..." or
"Skipping file, Not enough free space on disk."

If we had an issue with checking the free disk space it would be there.
Comment 6 Alex Newman 2015-11-02 10:47:34 EST
The test appliance from above is no longer available. I have checked on a different appliance.

No file exist at /var/www/miq/vmdb/appliance_console.log

[root@host-192-168-55-119 log]# pwd
/var/www/miq/vmdb/log
[root@host-192-168-55-119 log]# ll
total 416800
drwxr-xr-x. 2 root root        90 Oct 29 05:38 apache
-rw-r--r--. 1 root root        66 Oct 27 18:14 api.log
-rw-r--r--. 1 root root       590 Oct 29 05:35 appliance_console.log
-rw-r--r--. 1 root root     32788 Nov  1 23:35 audit.log
-rw-r--r--. 1 root root  35822268 Nov  2 10:40 automation.log
-rw-r--r--. 1 root root     22432 Nov  2 08:52 aws.log
-rw-r--r--. 1 root root 231470492 Nov  2 10:43 evm.log
-rw-r--r--. 1 root root        66 Oct 27 18:14 fog.log
-rw-r--r--. 1 root root      8878 Oct 31 09:36 gem_list.txt
-rw-r--r--. 1 root root        66 Oct 27 18:14 kubernetes.log
-rw-r--r--. 1 root root     61075 Oct 31 09:36 last_startup.txt
-rw-r--r--. 1 root root    224603 Nov  2 10:30 miq_ntpdate.log
-rw-r--r--. 1 root root        90 Oct 31 09:36 package_list_rpm.txt
-rw-r--r--. 1 root root   1941787 Nov  1 16:47 policy.log
-rw-r--r--. 1 root root  13422098 Nov  2 10:42 production.log
-rw-r--r--. 1 root root  30510228 Nov  2 10:43 rhevm.log
-rw-r--r--. 1 root root        66 Oct 27 18:14 scvmm.log
-rw-r--r--. 1 root root  68869312 Nov  2 10:43 top_output.log
-rw-r--r--. 1 root root        66 Oct 27 18:14 vim.log
-rw-r--r--. 1 root root    253912 Nov  2 10:42 vmstat_output.log
[root@host-192-168-55-119 log]# cat appliance_console.log
# Logfile created on 2015-10-29 05:31:57 -0400 by logger.rb/47272
I, [2015-10-29T05:31:57.527478 #6800]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#initialize_postgresql) : starting
I, [2015-10-29T05:32:38.743973 #6800]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#initialize_postgresql) : complete
I, [2015-10-29T05:32:39.108997 #6800]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#create_region) : starting
I, [2015-10-29T05:35:36.378738 #6800]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#create_region) : complete
[root@host-192-168-55-119 log]# uptime
 10:43:28 up 2 days,  2:08,  1 user,  load average: 0.34, 0.37, 0.34
[root@host-192-168-55-119 log]#
Comment 8 CFME Bot 2015-11-04 11:10:12 EST
New commit detected on ManageIQ/manageiq-appliance/master:
https://github.com/ManageIQ/manageiq-appliance/commit/e8a7495f3617ad553df9eb1436edd84d7f41ed7f

commit e8a7495f3617ad553df9eb1436edd84d7f41ed7f
Author:     Nick Carboni <ncarboni@redhat.com>
AuthorDate: Tue Nov 3 11:14:17 2015 -0500
Commit:     Nick Carboni <ncarboni@redhat.com>
CommitDate: Tue Nov 3 11:14:17 2015 -0500

    Run pre-rotate script even if source command fails
    
    There are some things that the various files sourced
    by /etc/default/evm do that SELinux will prevent based
    on the context logrotate runs under. Because of this
    we need to source the file in order to set our
    environment variables and run the pre-rotate script
    regardless of the exit status of the source command.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1265463

 COPY/etc/logrotate.d/miq_logs.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 10 Alex Newman 2015-11-09 09:35:06 EST
Version: 5.5.0.9-beta2.20151102161742_5530c9a 
 

[root@10-16-7-64 log]# pwd
/var/www/miq/vmdb/log
[root@10-16-7-64 log]# uptime
 09:23:37 up 3 days, 23:41,  1 user,  load average: 0.23, 0.22, 0.18
[root@10-16-7-64 log]# ll
total 214412
drwxr-xr-x. 2 root root        90 Nov  3 20:20 apache
-rw-r--r--. 1 root root        66 Nov  3 11:21 api.log
-rw-r--r--. 1 root root       594 Nov  3 20:15 appliance_console.log
-rw-r--r--. 1 root root     11483 Nov  3 20:16 audit.log
-rw-r--r--. 1 root root     32263 Nov  5 09:46 automation.log
-rw-r--r--. 1 root root        66 Nov  3 11:21 aws.log
-rw-r--r--. 1 root root 100920509 Nov  9 09:23 evm.log
-rw-r--r--. 1 root root        66 Nov  3 11:21 fog.log
-rw-r--r--. 1 root root      9024 Nov  5 09:44 gem_list.txt
-rw-r--r--. 1 root root        66 Nov  3 11:21 kubernetes.log
-rw-r--r--. 1 root root     60127 Nov  5 09:44 last_startup.txt
-rw-r--r--. 1 root root      1039 Nov  5 09:46 miq_ntpdate.log
-rw-r--r--. 1 root root        90 Nov  5 09:44 package_list_rpm.txt
-rw-r--r--. 1 root root      1607 Nov  5 09:46 policy.log
-rw-r--r--. 1 root root     35038 Nov  5 09:46 production.log
-rw-r--r--. 1 root root        66 Nov  3 11:21 rhevm.log
-rw-r--r--. 1 root root        66 Nov  3 11:21 scvmm.log
-rw-r--r--. 1 root root 115864294 Nov  9 09:23 top_output.log
-rw-r--r--. 1 root root        66 Nov  3 11:21 vim.log
-rw-r--r--. 1 root root    483551 Nov  9 09:23 vmstat_output.log
[root@10-16-7-64 log]# cat appliance_console.log
# Logfile created on 2015-11-03 20:14:11 -0500 by logger.rb/47272
I, [2015-11-03T20:14:11.669854 #10379]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#initialize_postgresql) : starting
I, [2015-11-03T20:14:26.919472 #10379]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#initialize_postgresql) : complete
I, [2015-11-03T20:14:27.442070 #10379]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#create_region) : starting
I, [2015-11-03T20:15:31.481287 #10379]  INFO -- : MIQ(ApplianceConsole::InternalDatabaseConfiguration#create_region) : complete
Comment 11 Nick Carboni 2015-11-09 09:56:18 EST
The build used for the test was from before the fix was included. When the bug gets included in a build the "Fixed In Version" field will contain the build version to use for testing this issue.
Comment 12 Alex Newman 2015-11-09 11:36:46 EST
Tested: 5.5.0.9-beta2.20151102161742_5530c9a
Comment 13 Alex Newman 2015-11-12 09:42:24 EST
Tested Version: 5.5.0.10-beta2.1.20151110134042_d6f5459 
Passed
Comment 15 errata-xmlrpc 2015-12-08 08:32:48 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:2551

Note You need to log in before you can comment on or make changes to this bug.