Description of problem: Deploy logging with "openshift_logging_fluentd_audit_container_engine=true", and enable remote rsyslog server, then check logs in rsyslog server and rsyslogback server, no docker event could be found. However, the docker event could be found in es pod, and other logs could be found in rsyslog server. Version-Release number of selected component (if applicable): oc v3.10.0-0.36.0 kubernetes v1.10.0+b81c8f8 features: Basic-Auth GSSAPI Kerberos SPNEGO openshift v3.10.0-0.36.0 kubernetes v1.10.0+b81c8f8 Image Version: logging-fluentd-v3.10.0-0.36.0.0 logging-elasticsearch-v3.10.0-0.36.0.2 Ansible Version: openshift-ansible-3.10.0-0.36.0.git.0.521f0ef.el7.noarch How reproducible: Always Steps to Reproduce: 1.Deploy logging with "openshift_logging_fluentd_audit_container_engine=true" 2.Enable remote rsyslog server 3.Execute docker commands 4.Check docker event in es and rsyslog server Actual results: Docker event could be found in es pod, but couldn't be found in rsyslog server. Expected results: Docker event could be found in es pod and rsyslog server when remote rsyslog server is enabled. Additional info: # oc get ds logging-fluentd -o yaml|grep -A 2 SYS - name: USE_REMOTE_SYSLOG value: "true" - name: REMOTE_SYSLOG_HOST value: $rsyslog_ip - name: REMOTE_SYSLOG_HOST_BACKUP value: $rsyslogback_ip - name: REMOTE_SYSLOG_PORT_BACKUP value: "514"
(In reply to Qiaoling Tang from comment #0) > Actual results: > Docker event could be found in es pod, but couldn't be found in rsyslog > server. Please attach the following data. 1) a couple of example docker events indexed in the ElasticSearch, 2) output from oc logs <fluentd_pod>, 3) the remote syslog config file in the fluentd pod. /etc/fluent/configs.d/dynamic/output-remote-syslog.conf Also, is ops enabled? What is missing in the rsyslog server is just audit log? Other logs are successfully forwarded? Thanks.
Hi Noriko, The ops wasn't enabled. The journal, docker log can be found in the rsyslog server.
Thank you, @Qiaoling! Looking at the config file [1], there is no tag_key directive here. There is a known issue [2] found by @Anping, which is supposed to be fixed [3] in the next build. Could you please rerun the test when the next build including the fix [3]? Thanks! [1] - https://bugzilla.redhat.com/attachment.cgi?id=1432991 [2] - https://bugzilla.redhat.com/show_bug.cgi?id=1515715#c18 [3] - https://github.com/openshift/origin-aggregated-logging/pull/1134
No log could be found in rsyslog server. Seems the changes are not in logging-fluentd/images/v3.10.1-1.
Verified on logging-fluentd/images/v3.10.7-1.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1816