Bug 1531295 - [RFE]: rsyslog imfile support for symlink wildcard combination
Summary: [RFE]: rsyslog imfile support for symlink wildcard combination
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rsyslog
Version: 7.4
Hardware: All
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: Jiří Vymazal
QA Contact: Noriko Hosoi
Mirek Jahoda
URL:
Whiteboard:
Keywords: FutureFeature, Patch, Triaged
: 1563347 1563409 (view as bug list)
Depends On: 1601928
Blocks: 1477664 1563347 1563409 1614179
TreeView+ depends on / blocked
 
Reported: 2018-01-04 23:00 UTC by Ruchika K
Modified: 2018-12-06 09:37 UTC (History)
17 users (show)

(edit)
*rsyslog* `imfile` now supports symlinks

With this update, the *rsyslog* `imfile` module delivers better performance and more configuration options. This enables to use the module for more complicated file monitoring use cases. Users of *rsyslog* are now able to use file monitors with glob patterns anywhere along the configured path and rotate symlink targets with increased data throughput when compared to the previous version.
Clone Of:
: 1563347 1614179 (view as bug list)
(edit)
Last Closed: 2018-10-30 10:17:00 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2018:3135 None None None 2018-10-30 10:19 UTC
Github rsyslog rsyslog issues 2229 None None None 2018-01-09 15:07 UTC
Github rsyslog rsyslog pull 2749 None None None 2018-06-01 15:01 UTC
Github rsyslog rsyslog pull 2914 None None None 2018-08-07 15:42 UTC
Github rsyslog rsyslog pull 2952 None None None 2018-08-24 13:58 UTC
Red Hat Bugzilla 1602156 None CLOSED imfile symlink uses source filename in metadata filename instead of given name 2019-04-25 15:37 UTC
Red Hat Bugzilla 1644547 None CLOSED [RFE] Implement directory wildcards in rsyslog 2019-04-25 15:37 UTC
Red Hat Bugzilla 1649250 None VERIFIED rsyslog-8.24.0-34.el7.x86_64 imfile broken with logrotate and tomcat rotated logs 2019-04-25 15:37 UTC

Internal Trackers: 1602156 1644547 1649250

Description Ruchika K 2018-01-04 23:00:19 UTC
Description of problem:
- rsyslog stops reading from files via a symlink.

Version-Release number of selected component (if applicable):

Redhat 7.4
rsyslog-8.24.0-12.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. configure rsyslog to read docker container logs (refer case linked)
2. rsyslog configured to data to a log database


Actual results:
rsyslog fails to see the change in the symlinked file.

Expected results:
rsyslog successfully reads the imfile by looking at what the symlink points to 

Additional info:

Please refer to the case linked for additional data.

Comment 2 Ruchika K 2018-01-04 23:12:04 UTC
This is the link describing the exact problem: https://github.com/rsyslog/rsyslog/issues/2229

Comment 3 Peter Portante 2018-01-04 23:16:35 UTC
The key to knowing if a fix is working is if rsyslog can be configured along side OCP 3.4 and later to watch /var/log/containers/*.log and see that logs continually flow as those log files are created and deleted (some with the same name as they roll over at the 50 MB limit).

Comment 13 Rich Megginson 2018-01-18 20:49:20 UTC
The problem is with rsyslog imfile inotify?

http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html

" mode ["inotify"/"polling"/"fen"]

        Default: “inotify”

        Available since: 8.1.5

    This specifies if imfile is shall run in inotify (“inotify”) or polling (“polling”) mode. Traditionally, imfile used polling mode, which is much more resource-intense (and slower) than inotify mode. It is suggested that users turn on “polling” mode only if they experience strange problems in inotify mode. In theory, there should never be a reason to enable “polling” mode and later versions will most probably remove it.
"

Has anyone tried to use `mode polling` as a workaround?  I think this qualifies as a "strange" problem.

Comment 14 Peter Portante 2018-01-19 01:54:33 UTC
I don't believe polling mode works with wildcards, see: http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#caveats-known-bugs

Comment 22 Jiří Vymazal 2018-04-04 13:36:08 UTC
*** Bug 1563409 has been marked as a duplicate of this bug. ***

Comment 23 Jiří Vymazal 2018-04-05 05:41:49 UTC
*** Bug 1563347 has been marked as a duplicate of this bug. ***

Comment 113 Anping Li 2018-09-04 17:30:09 UTC
The imfile work well with the symlink wildcard. 
input(type="imfile" file="/var/log/containers/*.log" tag="kubernetes" addmetadata="on")

rsyslog-8.24.0-34.el7.x86_64
rsyslog-mmjsonparse-8.24.0-34.el7.x86_64
rsyslog-mmnormalize-8.24.0-34.el7.x86_64
rsyslog-elasticsearch-8.24.0-34.el7.x86_64
rsyslog-mmkubernetes-8.24.0-34.el7.x86_64

Comment 117 errata-xmlrpc 2018-10-30 10:17:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:3135


Note You need to log in before you can comment on or make changes to this bug.