Bug 1649250
| Summary: | rsyslog-8.24.0-34.el7.x86_64 imfile broken with logrotate and tomcat rotated logs | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Jarno Huuskonen <jarno.huuskonen> |
| Component: | rsyslog | Assignee: | Jiří Vymazal <jvymazal> |
| Status: | CLOSED ERRATA | QA Contact: | Radovan Sroka <rsroka> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.6 | CC: | adam.winberg, dapospis, doug.wussler, fkrska, jvymazal, kompastver, pasik, rmeggins, rsroka, s.buesing, sjayapra, stturner |
| Target Milestone: | rc | Keywords: | Regression, Triaged |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | rsyslog-8.24.0-35.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-08-06 12:48:13 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1648377, 1649308, 1656860 | ||
*** Bug 1649491 has been marked as a duplicate of this bug. *** I built rsyslog-8.24.0-34.el7.x86_64 with this quick hack patch (manually copied/edited from https://github.com/rsyslog/rsyslog/commit/e929ee7220de8a81a8669573ac44526397bb3158). I'm testing this with tomcat access_log and it looks like rsyslog continues to monitor the access_log after inode change(rotation). So I think backporting the rsyslog pull/3051 should fix this problem. diff -urN rsyslog-8.24.0.orig/plugins/imfile/imfile.c rsyslog-8.24.0/plugins/imfile/imfile.c --- rsyslog-8.24.0.orig/plugins/imfile/imfile.c 2018-11-13 11:36:57.862001818 +0200 +++ rsyslog-8.24.0/plugins/imfile/imfile.c 2018-11-13 11:36:26.706026617 +0200 @@ -629,8 +629,11 @@ act_obj_unlink(act); restart = 1; break; + } else if(fileInfo.st_ino != act->ino) { + act_obj_unlink(act); + restart = 1; + break; } - // TODO: add inode check for change notification! /* Note: active nodes may get deleted, so we need to do the * pointer advancement at the end of the for loop! Any news on this? Just looking to have my expectations managed. The Priority and Severity of this issue are both HIGH. We are approaching 7 months since this issue was opened. Is the release of the fix imminent? The fix is targetting RHEL-7.7. What about RHEL8 rsyslog-8.37.0-9.el8 ? AFAIK 1674471 is tracking RHEL8 rsyslog, any update when rsyslog-8.37.0-10.el8 (for RHEL8) is going to be available ? Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:2110 |
Description of problem: After rhel7 update 6 (rsyslog-8.24.0-34.el7.x86_64) rsyslog imfile stopped working: - when tomcat rotates for example access_log ( <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="access_log" rotatable="true" renameOnRotate="true" requestAttributesEnabled="true" buffered="true" fileDateFormat=".yyyyMMdd" pattern="%h %l %u %t "%r" %s %b %D JSESSIONID=%{JSESSIONID}c" />) - sometimes when logrotate rotates apache access_log (doesn't happen always) Imfile example config for tomcat: module(load="imfile" mode="inotify" PollingInterval="20") input(type="imfile" ruleset="uef_imfile_fwd" File="/var/log/tomcat/access_log" PersistStateInterval="300" Tag="access-hostnamehere" Severity="info" Facility="local7" ) Imfile example for apache: module(load="imfile" mode="inotify" PollingInterval="20") input(type="imfile" ruleset="uef_imfile_fwd" File="/var/log/httpd/access_log" PersistStateInterval="300" Tag="access-hostnamehere" Severity="info" Facility="local7" ) (uef_imfile_fwd ruleset forwards logs to remote server). Version-Release number of selected component (if applicable): syslog-8.24.0-34.el7.x86_64 How reproducible: - Tomcat/java rotated logs: seems to happen after every rotate - logrotate/apache: happens sometimes Steps to Reproduce: 1. use rsyslog imfile to monitor tomcat access_log 2. let tomcat rotate logfile 3. rsyslog stops forwarding logs after Actual results: For example after after apache logrotate, rsyslog was monitoring wrong file: these are the files from /var/lib/rsyslog/ file: imfile-state:4237564 content: { "filename": "\/var\/log\/httpd\/access_log", "prev_was_nl": 0, "curr_offs": 2295391, "strt_offs": 2295391 } file: imfile-state:4331555 content: { "filename": "\/var\/log\/httpd\/error_log", "prev_was_nl": 0, "curr_offs": 1242, "strt_offs": 1242 } Actual /var/log/httpd/access_log was already using different inode: File: ‘/var/log/httpd/access_log’ Size: 255753 Blocks: 512 IO Block: 4096 regular file Device: fd03h/64771d Inode: 4237513 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: system_u:object_r:httpd_log_t:s0 Access: 2018-11-11 04:02:02.936520161 +0200 Modify: 2018-11-12 10:09:36.723540441 +0200 Change: 2018-11-12 10:09:36.723540441 +0200 So AFAIK rsyslog(imfile) didn't recognise rotation. Similar situation with tomcat access_log: /var/lib/rsyslog/imfile-state:164 ({ "filename": "\/var\/log\/tomcat\/access_log", "prev_was_nl": 0, "curr_offs": 83250, "strt_offs": 83250 }) while the actual logfile is already using different inode: File: ‘/var/log/tomcat/access_log’ Size: 38850 Blocks: 80 IO Block: 4096 regular file Device: fd07h/64775d Inode: 190 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 91/ tomcat) Gid: ( 91/ tomcat) Context: system_u:object_r:tomcat_log_t:s0 Access: 2018-11-13 10:15:54.450811462 +0200 Modify: 2018-11-13 10:28:30.741826028 +0200 Change: 2018-11-13 10:28:30.741826028 +0200 Expected results: rsyslog(imfile) continues to monitor/forward logs after logrotation. Additional info: Workaround for tomcat access logs: When using logrotate+copytruncate for tomcat accesslog(+rotatable="false") (and reloadontruncate="on" with imfile) seems to work. This sounds like the problem described in rsyslog bug 3051 (https://github.com/rsyslog/rsyslog/pull/3051 / https://github.com/rsyslog/rsyslog/commit/e929ee7220de8a81a8669573ac44526397bb3158) AFAIK rsyslog-8.24.0-34.el7.x86_64 imfile is missing the above commit.