Description of problem: Using docker json-file, and there are several projects where logs have already been collected. There is already a /var/log/es-containers.log.pos file which contains the position of the last log read from the json log files from the containers in the various projects. Configure throttling for one or more of the projects Throttling will reuse the same /var/log/es-containers.log.pos for the new in_tail configuration for the throttled projects. This is a no-no: https://docs.fluentd.org/v0.12/articles/in_tail#posfile-highly-recommended "Don't share pos_file between in_tail configurations. It causes unexpected behavior, e.g. corrupt pos_file content." Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
https://github.com/openshift/origin-aggregated-logging/pull/803
Blocked by Bug 1532955
Should there is a pos file for each container under /var/log/? Should es-containers.log.pos be deleted? The following pos file are present during v3.9 testing. openshift3/logging-elasticsearch:v3.9.0-0.20.0.0 openshift-elasticsearch-plugin-2.4.4.17__redhat_1-3.el7.noarch # ls /var/log/| grep pos es-containers.log.pos journal.pos
No, there may still be a containers.log.pos file that exists, however if you are throttling a specific namespace you should not see any entries for that namespace in the containers.log.pos file. To test this you will need to configure log throttling for Fluentd. You should see a pos file for each throttled namespace you have configured.
How could I throttle a specific namespace? could you give me a example?
Verified in openshift3/logging-fluentd/images/v3.9.0-0.21.0.0 Once throttle a specific namespace, The pos file is created and the pos pointer are move to this file from es-containers.log.pos -rw-r--r--. 1 root root 1199 Jan 18 05:13 es-container-logging.log.pos -rw-r--r--. 1 root root 164 Jan 18 05:14 es-container-openshift-operations.log.pos -rw-r--r--. 1 root root 2430 Jan 18 05:13 es-containers.log.pos
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489