RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1303617 - [RFE] wildcard filename support
Summary: [RFE] wildcard filename support
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rsyslog
Version: 7.2
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: Peter Vrabec
QA Contact: Stefan Dordevic
Mirek Jahoda
URL:
Whiteboard:
Depends On:
Blocks: 1203710 1296594 1313485
TreeView+ depends on / blocked
 
Reported: 2016-02-01 13:11 UTC by Rich Megginson
Modified: 2021-03-11 14:28 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
_rsyslog_ imfile module now supports a wildcard file name The _rsyslog_ packages provide an enhanced, multi-threaded syslog daemon. With this update, the _rsyslog_ imfile module supports using wildcards inside file names and adding the actual file name to the message's metadata. This is useful, when _rsyslog_ needs to read logs under a directory and does not know the names of files in advance.
Clone Of:
Environment:
Last Closed: 2016-11-04 05:37:23 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Backport of wildcard support (71.68 KB, patch)
2016-05-03 14:50 UTC, Radovan Sroka 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2401 0 normal SHIPPED_LIVE rsyslog bug fix and enhancement update 2016-11-03 13:55:50 UTC

Description Rich Megginson 2016-02-01 13:11:36 UTC 
Description of problem:

openshift needs to read log files under /var/log/containers - one log per container.  This is difficult to do with rsyslog v7 imfile because you don't know the names of the files in advance, and even if you did, there could be thousands of them that change frequently.  It would be much easier if you could just configure imfile to read from /var/log/containers/*, and put the filename in the message metadata.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html?highlight=wildcard#

v8 (version 8.5.0 and later) with inotify support can have imfile use a wildcard filename and add the actual filename to the metadata.  I don't know if it will be possible to backport this functionality to v7.
Comment 2 Rich Megginson 2016-02-03 14:54:13 UTC 
After speaking with Tomas, it may be possible to backport this feature from rsyslog v8 to rsyslog v7 in RHEL 7.3.  I am requesting acks for 7.3.
Comment 8 Radovan Sroka 2016-05-03 14:50:25 UTC 
Created attachment 1153459 [details]
Backport of wildcard support

This backport introduce some new features:

Module:
        - option "mode" which can be polling|inotify and it's not required
        - !BUT! there is difference between us and upstream
        - we have as default option "polling" and upstream has "inotify"
        - because we need to preserve backward compatibility

Input:
        - new options: escapeLF, removestateondelete works as upstream 
        - new features: dynamic statefiles, wildcards(inotify mode)
        - statefile is not required now, if it's not given then dynamic statefile is created

Refactoring of internal structures:
        - active file table: from fixed array to double-linked list 
        - using listners as upstream

doc/imfile.html is updated as well.

This patch was derived from upstream commits:
        ea9c81ce
        7ca5b1e5
        c9c6bd47
        68ac87cb
        25baf0e1
        dcd19358
        36f373be
        d11858de
        fd7ab1ee
        f63d5ed4
        e73b8413
        7cf1349a
        e53d6114
        ef9fad05
        e5e7836b
        385fde74
        0f164bf7

---------------------------------------------------------------------

And here's some test cases:
---

#default is polling mode so it's ok

module(load="imfile")
input(type="imfile" tag="tagggg1" file="/tmp/input_file")
input(type="imfile" tag="tagggg2" file="/tmp/input_data")

------------------------------------------------------------------
#default is polling so wildcards are not allowed

module(load="imfile")
input(type="imfile" tag="tagggg1" file="/tmp/input_*")

-----------------------------------------------------------------
#it's monitoring all matched files

module(load="imfile" mode="inotify")
input(type="imfile" tag="tagggg1" file="/tmp/input_*")

-----------------------------------------------------------------
#test of polling mode

module(load="imfile" mode="polling")
input(type="imfile" tag="tagggg1" file="/tmp/input_file")
input(type="imfile" tag="tagggg2" file="/tmp/input_data")

-----------------------------------------------------------------
#in polling mode wildcards are not allowed

module(load="imfile" mode="polling")
input(type="imfile" tag="tagggg1" file="/tmp/input_*")

-----------------------------------------------------------------
#statefile in inotify mode without wildcards is ok 

module(load="imfile" mode="inotify")
input(type="imfile" tag="tagggg1" statefile="st1" file="/tmp/input_file")
input(type="imfile" tag="tagggg2" statefile="st2" file="/tmp/input_data")

-----------------------------------------------------------------
#statefile in polling mode is ok

module(load="imfile" mode="polling")
input(type="imfile" tag="tagggg1" statefile="st1" file="/tmp/input_file")
input(type="imfile" tag="tagggg2" statefile="st2" file="/tmp/input_data")

-----------------------------------------------------------------
#statefiles with wildcards are not allowed

module(load="imfile" mode="inotify")
input(type="imfile" tag="tagggg1" statefile="st1" file="/tmp/input_*")
Comment 16 errata-xmlrpc 2016-11-04 05:37:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2401.html
Note You need to log in before you can comment on or make changes to this bug.