Bug 1531295

Summary: [RFE]: rsyslog imfile support for symlink wildcard combination
Product: Red Hat Enterprise Linux 7 Reporter: Ruchika K <rkharwar>
Component: rsyslogAssignee: Jiří Vymazal <jvymazal>
Status: CLOSED ERRATA QA Contact: Noriko Hosoi <nhosoi>
Severity: urgent Docs Contact: Mirek Jahoda <mjahoda>
Priority: urgent    
Version: 7.4CC: anli, cww, dapospis, dkopecek, jvymazal, ksrot, kwalker, lmiksik, mjahoda, mthacker, nhosoi, pasik, pportant, rkharwar, rmeggins, schituku, tkatarki
Target Milestone: rcKeywords: FutureFeature, Patch, Triaged
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: rsyslog-8.24.0-34.el7 Doc Type: Enhancement
Doc Text:
*rsyslog* `imfile` now supports symlinks With this update, the *rsyslog* `imfile` module delivers better performance and more configuration options. This enables to use the module for more complicated file monitoring use cases. Users of *rsyslog* are now able to use file monitors with glob patterns anywhere along the configured path and rotate symlink targets with increased data throughput when compared to the previous version.
Story Points: ---
Clone Of:
: 1563347 1614179 (view as bug list) Environment:
Last Closed: 2018-10-30 10:17:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1601928    
Bug Blocks: 1477664, 1563347, 1563409, 1614179    

Description Ruchika K 2018-01-04 23:00:19 UTC
Description of problem:
- rsyslog stops reading from files via a symlink.

Version-Release number of selected component (if applicable):

Redhat 7.4
rsyslog-8.24.0-12.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. configure rsyslog to read docker container logs (refer case linked)
2. rsyslog configured to data to a log database


Actual results:
rsyslog fails to see the change in the symlinked file.

Expected results:
rsyslog successfully reads the imfile by looking at what the symlink points to 

Additional info:

Please refer to the case linked for additional data.

Comment 2 Ruchika K 2018-01-04 23:12:04 UTC
This is the link describing the exact problem: https://github.com/rsyslog/rsyslog/issues/2229

Comment 3 Peter Portante 2018-01-04 23:16:35 UTC
The key to knowing if a fix is working is if rsyslog can be configured along side OCP 3.4 and later to watch /var/log/containers/*.log and see that logs continually flow as those log files are created and deleted (some with the same name as they roll over at the 50 MB limit).

Comment 13 Rich Megginson 2018-01-18 20:49:20 UTC
The problem is with rsyslog imfile inotify?

http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html

" mode ["inotify"/"polling"/"fen"]

        Default: “inotify”

        Available since: 8.1.5

    This specifies if imfile is shall run in inotify (“inotify”) or polling (“polling”) mode. Traditionally, imfile used polling mode, which is much more resource-intense (and slower) than inotify mode. It is suggested that users turn on “polling” mode only if they experience strange problems in inotify mode. In theory, there should never be a reason to enable “polling” mode and later versions will most probably remove it.
"

Has anyone tried to use `mode polling` as a workaround?  I think this qualifies as a "strange" problem.

Comment 14 Peter Portante 2018-01-19 01:54:33 UTC
I don't believe polling mode works with wildcards, see: http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#caveats-known-bugs

Comment 22 Jiří Vymazal 2018-04-04 13:36:08 UTC
*** Bug 1563409 has been marked as a duplicate of this bug. ***

Comment 23 Jiří Vymazal 2018-04-05 05:41:49 UTC
*** Bug 1563347 has been marked as a duplicate of this bug. ***

Comment 113 Anping Li 2018-09-04 17:30:09 UTC
The imfile work well with the symlink wildcard. 
input(type="imfile" file="/var/log/containers/*.log" tag="kubernetes" addmetadata="on")

rsyslog-8.24.0-34.el7.x86_64
rsyslog-mmjsonparse-8.24.0-34.el7.x86_64
rsyslog-mmnormalize-8.24.0-34.el7.x86_64
rsyslog-elasticsearch-8.24.0-34.el7.x86_64
rsyslog-mmkubernetes-8.24.0-34.el7.x86_64

Comment 117 errata-xmlrpc 2018-10-30 10:17:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:3135