Bug 1531295 - [RFE]: rsyslog imfile support for symlink wildcard combination
Summary: [RFE]: rsyslog imfile support for symlink wildcard combination
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rsyslog
Version: 7.4
Hardware: All
OS: Unspecified
Target Milestone: rc
: ---
Assignee: Jiří Vymazal
QA Contact: Noriko Hosoi
Mirek Jahoda
: 1563347 1563409 (view as bug list)
Depends On: 1601928
Blocks: 1477664 1563347 1563409 1614179
TreeView+ depends on / blocked
Reported: 2018-01-04 23:00 UTC by Ruchika K
Modified: 2018-12-06 09:37 UTC (History)
17 users (show)

Fixed In Version: rsyslog-8.24.0-34.el7
Doc Type: Enhancement
Doc Text:
*rsyslog* `imfile` now supports symlinks With this update, the *rsyslog* `imfile` module delivers better performance and more configuration options. This enables to use the module for more complicated file monitoring use cases. Users of *rsyslog* are now able to use file monitors with glob patterns anywhere along the configured path and rotate symlink targets with increased data throughput when compared to the previous version.
Clone Of:
: 1563347 1614179 (view as bug list)
Last Closed: 2018-10-30 10:17:00 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github rsyslog rsyslog issues 2229 0 None closed make imfile support symlinks in monitored directories (requires some polling) 2020-11-12 01:58:43 UTC
Github rsyslog rsyslog pull 2749 0 None closed Symlink support for imfile 2020-11-12 01:58:24 UTC
Github rsyslog rsyslog pull 2914 0 None closed Enhanced symlink rotation support 2020-11-12 01:58:24 UTC
Github rsyslog rsyslog pull 2952 0 None closed Fixed parent name when processing symlinks 2020-11-12 01:58:24 UTC
Red Hat Bugzilla 1602156 0 unspecified CLOSED imfile symlink uses source filename in metadata filename instead of given name 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1644547 0 unspecified CLOSED [RFE] Implement directory wildcards in rsyslog 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1649250 0 high CLOSED rsyslog-8.24.0-34.el7.x86_64 imfile broken with logrotate and tomcat rotated logs 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHEA-2018:3135 0 None None None 2018-10-30 10:19:08 UTC

Internal Links: 1602156 1644547 1649250

Description Ruchika K 2018-01-04 23:00:19 UTC
Description of problem:
- rsyslog stops reading from files via a symlink.

Version-Release number of selected component (if applicable):

Redhat 7.4

How reproducible:

Steps to Reproduce:
1. configure rsyslog to read docker container logs (refer case linked)
2. rsyslog configured to data to a log database

Actual results:
rsyslog fails to see the change in the symlinked file.

Expected results:
rsyslog successfully reads the imfile by looking at what the symlink points to 

Additional info:

Please refer to the case linked for additional data.

Comment 2 Ruchika K 2018-01-04 23:12:04 UTC
This is the link describing the exact problem: https://github.com/rsyslog/rsyslog/issues/2229

Comment 3 Peter Portante 2018-01-04 23:16:35 UTC
The key to knowing if a fix is working is if rsyslog can be configured along side OCP 3.4 and later to watch /var/log/containers/*.log and see that logs continually flow as those log files are created and deleted (some with the same name as they roll over at the 50 MB limit).

Comment 13 Rich Megginson 2018-01-18 20:49:20 UTC
The problem is with rsyslog imfile inotify?


" mode ["inotify"/"polling"/"fen"]

        Default: “inotify”

        Available since: 8.1.5

    This specifies if imfile is shall run in inotify (“inotify”) or polling (“polling”) mode. Traditionally, imfile used polling mode, which is much more resource-intense (and slower) than inotify mode. It is suggested that users turn on “polling” mode only if they experience strange problems in inotify mode. In theory, there should never be a reason to enable “polling” mode and later versions will most probably remove it.

Has anyone tried to use `mode polling` as a workaround?  I think this qualifies as a "strange" problem.

Comment 14 Peter Portante 2018-01-19 01:54:33 UTC
I don't believe polling mode works with wildcards, see: http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#caveats-known-bugs

Comment 22 Jiří Vymazal 2018-04-04 13:36:08 UTC
*** Bug 1563409 has been marked as a duplicate of this bug. ***

Comment 23 Jiří Vymazal 2018-04-05 05:41:49 UTC
*** Bug 1563347 has been marked as a duplicate of this bug. ***

Comment 113 Anping Li 2018-09-04 17:30:09 UTC
The imfile work well with the symlink wildcard. 
input(type="imfile" file="/var/log/containers/*.log" tag="kubernetes" addmetadata="on")


Comment 117 errata-xmlrpc 2018-10-30 10:17:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.