1478821 – Fluentd log is filled up with warnings about "log unreadable. It is excluded and would be examined next time." when using json-file log driver

Bug 1478821 - Fluentd log is filled up with warnings about "log unreadable. It is excluded and would be examined next time." when using json-file log driver

Summary: Fluentd log is filled up with warnings about "log unreadable. It is excluded ...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Logging
Sub Component:
Version:	3.6.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	low
Severity:	low
Target Milestone:	---
Target Release:	3.6.z
Assignee:	Jeff Cantrill
QA Contact:	Xia Zhao
Docs Contact:
URL:
Whiteboard:
Depends On:	1477787
Blocks:	1481364
TreeView+	depends on / blocked

Reported:	2017-08-07 08:01 UTC by Xia Zhao
Modified:	2020-12-14 09:22 UTC (History)
CC List:	10 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	Cause: Consequence: Fix: Add exclude pattern to configuration to exclude messages with this priority level Result: Message is no longer visible
Clone Of:
Environment:
Last Closed:	2017-10-25 13:04:36 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
fluentd_log_#2 (38.75 KB, text/plain) 2017-08-07 08:01 UTC, Xia Zhao	no flags	Details
fluentd_log_#1 (348.40 KB, text/plain) 2017-08-07 08:01 UTC, Xia Zhao	no flags	Details
es log (13.00 KB, text/plain) 2017-08-07 08:01 UTC, Xia Zhao	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2017:3049	0	normal	SHIPPED_LIVE	OpenShift Container Platform 3.6, 3.5, and 3.4 bug fix and enhancement update	2017-10-25 15:57:15 UTC

Description Xia Zhao 2017-08-07 08:01:02 UTC

Created attachment 1309952 [details]
fluentd_log_#2

Description of problem:
Deploy logging 3.6 stacks on OCP env with json-file docker log driver. Wait for a while after EFK pods are started up, check the fluentd log, it is filled up with warnings like this:

2017-08-07 03:20:50 -0400 [warn]: /var/log/containers/java-mainclass-1-7hz6s_java_java-mainclass-bd852c77e9ddba7bf090aa2070fd325939ec6f622834c9553f301120c768ea52.log unreadable. It is excluded and would be examined next time.

Checked on node that the files/linked files should be accessible (file right is set to 777):
# ls -l /var/log/containers/java-mainclass-1-7hz6s_java_java-mainclass-bd852c77e9ddba7bf090aa2070fd325939ec6f622834c9553f301120c768ea52.log
lrwxrwxrwx. 1 root root 71 8月   7 03:07 /var/log/containers/java-mainclass-1-7hz6s_java_java-mainclass-bd852c77e9ddba7bf090aa2070fd325939ec6f622834c9553f301120c768ea52.log -> /var/log/pods/02ef8522-7b3f-11e7-b1f4-fa163e8691cf/java-mainclass_0.log

# ls -l /var/log/pods/02ef8522-7b3f-11e7-b1f4-fa163e8691cf/java-mainclass_0.log
lrwxrwxrwx. 1 root root 165 8月   7 03:07 /var/log/pods/02ef8522-7b3f-11e7-b1f4-fa163e8691cf/java-mainclass_0.log -> /var/lib/docker/containers/bd852c77e9ddba7bf090aa2070fd325939ec6f622834c9553f301120c768ea52/bd852c77e9ddba7bf090aa2070fd325939ec6f622834c9553f301120c768ea52-json.log


Version-Release number of selected component (if applicable):
logging-fluentd                           v3.6.173.0.5-3      81d989e6ab3f        3 days ago          232 MB

How reproducible:
Always

Steps to Reproduce:
1.Deploy logging 3.6 stacks on OCP env with json-file docker log driver
2.Wait for a while after EFK pods are started up, check the fluentd log

Actual results:
Fluentd log is filled up with warnings about "log unreadable. It is excluded and would be examined next time."

Expected results:
Fluentd log should not be filled up with warnings about "log unreadable. It is excluded and would be examined next time."

Additional info:
1. fluent and es logs are attached.
2. Issue did not repro to journald log driver

Comment 1 Xia Zhao 2017-08-07 08:01:29 UTC

Created attachment 1309953 [details]
fluentd_log_#1

Comment 2 Xia Zhao 2017-08-07 08:01:50 UTC

Created attachment 1309955 [details]
es log

Comment 3 Eduardo Minguez 2017-08-07 12:17:09 UTC

It seems to affect OCP 3.4 as well:

[cloud-user@bastion ~]$ oc version
oc v3.4.1.44
kubernetes v1.4.0+776c994
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ocp.edu.example.com:8443
openshift v3.4.1.44
kubernetes v1.4.0+776c994

[root@app-node-2 ~]# rpm -qa | grep -i openshift
openshift-ansible-3.4.119-1.git.0.2b36c8a.el7.noarch
openshift-ansible-roles-3.4.119-1.git.0.2b36c8a.el7.noarch
atomic-openshift-3.4.1.44-1.git.0.e98176f.el7.x86_64
atomic-openshift-excluder-3.4.1.44-1.git.0.e98176f.el7.noarch
atomic-openshift-docker-excluder-3.4.1.44-1.git.0.e98176f.el7.noarch
openshift-ansible-docs-3.4.119-1.git.0.2b36c8a.el7.noarch
openshift-ansible-filter-plugins-3.4.119-1.git.0.2b36c8a.el7.noarch
openshift-ansible-callback-plugins-3.4.119-1.git.0.2b36c8a.el7.noarch
openshift-ansible-playbooks-3.4.119-1.git.0.2b36c8a.el7.noarch
atomic-openshift-clients-3.4.1.44-1.git.0.e98176f.el7.x86_64
atomic-openshift-node-3.4.1.44-1.git.0.e98176f.el7.x86_64
openshift-ansible-lookup-plugins-3.4.119-1.git.0.2b36c8a.el7.noarch
atomic-openshift-utils-3.4.119-1.git.0.2b36c8a.el7.noarch
tuned-profiles-atomic-openshift-node-3.4.1.44-1.git.0.e98176f.el7.x86_64

[cloud-user@bastion ~]$ oc get pods
NAME                          READY     STATUS      RESTARTS   AGE
logging-curator-1-vcsfq       1/1       Running     0          1h
logging-deployer-kxxgh        0/1       Completed   0          1h
logging-es-8bnwfrsq-1-zi1cl   1/1       Running     0          1h
logging-es-ps2gwwso-1-qathy   1/1       Running     0          1h
logging-es-swdwnrcl-1-ec2oo   1/1       Running     0          1h
logging-fluentd-1togp         1/1       Running     0          53m
logging-fluentd-esl4p         1/1       Running     0          54m
logging-fluentd-hygkb         1/1       Running     0          54m
logging-fluentd-s0gmf         1/1       Running     0          54m
logging-fluentd-tmiel         1/1       Running     0          54m
logging-fluentd-to3wi         1/1       Running     0          54m
logging-kibana-1-ce07t        2/2       Running     1          1h


[cloud-user@bastion ~]$ oc logs logging-fluentd-s0gmf
umounts of dead containers will fail. Ignoring...
umount: /var/lib/docker/containers/*/shm: mountpoint not found
2017-08-07 07:21:51 -0400 [info]: reading config file path="/etc/fluent/fluent.conf"
2017-08-07 07:22:00 -0400 [warn]: /var/log/containers/logging-deployer-kxxgh_logging_POD-c2d96d5bd64f4432fea176e3f7eb382bb3b72d927f0d3406e1292e439747ef21.log unreadable. It is excluded and would be examined next time.
2017-08-07 07:22:00 -0400 [warn]: /var/log/containers/logging-es-swdwnrcl-1-ec2oo_logging_elasticsearch-796ade94d1096d7acaf31cac89a6907f7e7effe874e8b846bb59e364d062887e.log unreadable. It is excluded and would be examined next time.
2017-08-07 07:22:00 -0400 [warn]: /var/log/containers/logging-fluentd-s0gmf_logging_fluentd-elasticsearch-4df4382ea7d99c333abf745577dda33c6f5e16b54e55054363e8696c57c7080c.log unreadable. It is excluded and would be examined next time.
2017-08-07 07:22:00 -0400 [warn]: /var/log/containers/logging-fluentd-7wj4e_logging_POD-4753aac9c86485017963d0e795a3c25187f030792a8aa257df411ddc66bd682e.log unreadable. It is excluded and would be examined next time.
...
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-es-swdwnrcl-1-ec2oo_logging_elasticsearch-796ade94d1096d7acaf31cac89a6907f7e7effe874e8b846bb59e364d062887e.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-fluentd-s0gmf_logging_fluentd-elasticsearch-4df4382ea7d99c333abf745577dda33c6f5e16b54e55054363e8696c57c7080c.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/skydive-agent-bivvj_skydive_skydive-agent-1debce5fc61a007a66f5eef6dc4d0865492a81c1348f050b7d11ae127dc1d728.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-fluentd-s0gmf_logging_POD-a4ac77f49176215e498182a8a7d82440ba53db37cfa92e393227e7f4d99bde13.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-es-swdwnrcl-1-ec2oo_logging_POD-549de4119b74944c4434f508d7165726836ead3267167b3a474a4e16cd427235.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-deployer-kxxgh_logging_deployer-946969b34f067d5aee9188f354f3070298174b1e32c31c7a382412677bf71a04.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/router-1-j8hfz_default_router-8a00745e16624724d4285e4d86d3a90831cc6cec1bc1e438584352c0f6224905.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-curator-1-vcsfq_logging_curator-36aa5a17a0da5c7b24c4b5718a3e9d2e85e936474727ce12664adb6e7c36884a.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/router-1-j8hfz_default_POD-2f57d772b48fe263eb8a5f7229131730b6d6b8bcaad07f3abe48e9a2dcea9b22.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/skydive-agent-bivvj_skydive_POD-53e631b6d8222618e500552a71fb981792af725f627eea14fd9e10b37eff6c5b.log unreadable. It is excluded and would be examined next time.
2017-08-07 08:16:00 -0400 [warn]: /var/log/containers/logging-curator-1-vcsfq_logging_POD-f60a3ebd1d24813347929dd69fd50a05c28774b327d7fa0d07898ef6da634666.log unreadable. It is excluded and would be examined next time.

Comment 4 Eduardo Minguez 2017-08-07 12:34:56 UTC

It looks like the /var/lib/docker/containers mountpoint is empty:

[cloud-user@bastion ~]$ oc describe pod logging-fluentd-dpo7t | grep -i lib
      /var/lib/docker/containers from varlibdockercontainers (ro)
  varlibdockercontainers:
    Path:	/var/lib/docker/containers
[cloud-user@bastion ~]$ oc get pods -o wide | grep dpo7t
logging-fluentd-dpo7t         1/1       Running     1          9m        172.30.91.4    infra-node-0.control.edu.example.com

[cloud-user@bastion ~]$ oc rsh logging-fluentd-dpo7t
sh-4.2# cd /var/lib/docker/containers/
sh-4.2# ls
sh-4.2# ls -la
total 0
drwxr-xr-x. 2 root root  6 Aug  7 08:29 .
drwxr-xr-x. 3 root root 24 Aug  7 08:29 ..

And in the infra-node-0:

15058ae63bbfe1333a43eaaca31bf11bcf80b6896170ea2ba561c353865f463b
1b0e42d3528d0d788395a42c3ee4e40e3d9f0989eea79f4042698ea5eccedf30
1eae6a3459557a21852dd2f41440f23303ec5ae26847da25f7f12fc4d6c1a70b
5619f6146280a948e581da1addf8f60422e3d01edb7f072d1d3bb562b2388115
7a87481a121cc26697c2e06ca3a111578f8218cae925e75a58ec73a534bf167c
8d73a14adc26d31f02102d5b0609ab73a811ba14a6876414b995ee2bf040a171
9871b1c7972ab136b02724b5fadc4234cba28e8c82260e4e1801ab285b322a3a
989f5db8514c446ea201d3c84ba2ea5a0f24f060ee4fc11bcd98579658c7fb22
9ad5005e624a7737c142f1907fb33350280634e7198a1936fae84fbbe542b658
9ff97b92b8ca87318d90ce487bd3f53e05a3b6319f028a11a6c513217d14e72b
c5cc7607583c553dd52bd11badd8ad31fc2e32f70f5f2269fdf808910bb20a35
d7e51149e2255445eff8288117bee868867a005fad520c4f7def54a7bab71a71
e3d5d457ee9e841ecc47539cf582875a3a0c88d49c81842dda248c6d11abe70a
[root@infra-node-0 containers]# find . -type f -iname *.log
./7a87481a121cc26697c2e06ca3a111578f8218cae925e75a58ec73a534bf167c/7a87481a121cc26697c2e06ca3a111578f8218cae925e75a58ec73a534bf167c-json.log
./c5cc7607583c553dd52bd11badd8ad31fc2e32f70f5f2269fdf808910bb20a35/c5cc7607583c553dd52bd11badd8ad31fc2e32f70f5f2269fdf808910bb20a35-json.log
./e3d5d457ee9e841ecc47539cf582875a3a0c88d49c81842dda248c6d11abe70a/e3d5d457ee9e841ecc47539cf582875a3a0c88d49c81842dda248c6d11abe70a-json.log
./989f5db8514c446ea201d3c84ba2ea5a0f24f060ee4fc11bcd98579658c7fb22/989f5db8514c446ea201d3c84ba2ea5a0f24f060ee4fc11bcd98579658c7fb22-json.log
./15058ae63bbfe1333a43eaaca31bf11bcf80b6896170ea2ba561c353865f463b/15058ae63bbfe1333a43eaaca31bf11bcf80b6896170ea2ba561c353865f463b-json.log
./d7e51149e2255445eff8288117bee868867a005fad520c4f7def54a7bab71a71/d7e51149e2255445eff8288117bee868867a005fad520c4f7def54a7bab71a71-json.log
./9ad5005e624a7737c142f1907fb33350280634e7198a1936fae84fbbe542b658/9ad5005e624a7737c142f1907fb33350280634e7198a1936fae84fbbe542b658-json.log
./1eae6a3459557a21852dd2f41440f23303ec5ae26847da25f7f12fc4d6c1a70b/1eae6a3459557a21852dd2f41440f23303ec5ae26847da25f7f12fc4d6c1a70b-json.log
./8d73a14adc26d31f02102d5b0609ab73a811ba14a6876414b995ee2bf040a171/8d73a14adc26d31f02102d5b0609ab73a811ba14a6876414b995ee2bf040a171-json.log
./9ff97b92b8ca87318d90ce487bd3f53e05a3b6319f028a11a6c513217d14e72b/9ff97b92b8ca87318d90ce487bd3f53e05a3b6319f028a11a6c513217d14e72b-json.log
./9871b1c7972ab136b02724b5fadc4234cba28e8c82260e4e1801ab285b322a3a/9871b1c7972ab136b02724b5fadc4234cba28e8c82260e4e1801ab285b322a3a-json.log
./5619f6146280a948e581da1addf8f60422e3d01edb7f072d1d3bb562b2388115/5619f6146280a948e581da1addf8f60422e3d01edb7f072d1d3bb562b2388115-json.log
./1b0e42d3528d0d788395a42c3ee4e40e3d9f0989eea79f4042698ea5eccedf30/1b0e42d3528d0d788395a42c3ee4e40e3d9f0989eea79f4042698ea5eccedf30-json.log


Restorecon shows log files are not properly flagged:

[root@app-node-2 ~]# ls -lZ /var/lib/docker/containers/
drwx------. root root system_u:object_r:container_var_lib_t:s0 0324acfa995d8d170fef350ac7c7fc083bbe3d102c8dbf614ac912676635e744
drwx------. root root system_u:object_r:container_var_lib_t:s0 a6216825dfcc9e20d55b5da9031a8e7fb776e64e4765b213c543d581b01a7c41
drwx------. root root system_u:object_r:container_var_lib_t:s0 e250508c319060139cbb26991b02157c44f3b04c6d48fc3d4238c65312c30124
drwx------. root root system_u:object_r:container_var_lib_t:s0 f2d8cbe979f4af4b4a1e95c983fc481c41dc439ae475b13645f1ba0bd4372c36
drwx------. root root system_u:object_r:container_var_lib_t:s0 f66543b868b8fc267cfbabfe2ac5a864771607708f4935370ca3431f3a762935
drwx------. root root system_u:object_r:container_var_lib_t:s0 fc4cac63b4234841bff4c37f8a078572015aee19d0b9ab405d5b2de3664ec958


[root@app-node-2 ~]# restorecon -Rv /var/lib/docker/containers/
restorecon reset /var/lib/docker/containers/a6216825dfcc9e20d55b5da9031a8e7fb776e64e4765b213c543d581b01a7c41/a6216825dfcc9e20d55b5da9031a8e7fb776e64e4765b213c543d581b01a7c41-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0
restorecon reset /var/lib/docker/containers/fc4cac63b4234841bff4c37f8a078572015aee19d0b9ab405d5b2de3664ec958/fc4cac63b4234841bff4c37f8a078572015aee19d0b9ab405d5b2de3664ec958-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0
restorecon reset /var/lib/docker/containers/f66543b868b8fc267cfbabfe2ac5a864771607708f4935370ca3431f3a762935/f66543b868b8fc267cfbabfe2ac5a864771607708f4935370ca3431f3a762935-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0
restorecon reset /var/lib/docker/containers/e250508c319060139cbb26991b02157c44f3b04c6d48fc3d4238c65312c30124/e250508c319060139cbb26991b02157c44f3b04c6d48fc3d4238c65312c30124-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0
restorecon reset /var/lib/docker/containers/f2d8cbe979f4af4b4a1e95c983fc481c41dc439ae475b13645f1ba0bd4372c36/f2d8cbe979f4af4b4a1e95c983fc481c41dc439ae475b13645f1ba0bd4372c36-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0
restorecon reset /var/lib/docker/containers/0324acfa995d8d170fef350ac7c7fc083bbe3d102c8dbf614ac912676635e744/0324acfa995d8d170fef350ac7c7fc083bbe3d102c8dbf614ac912676635e744-json.log context system_u:object_r:container_var_lib_t:s0->system_u:object_r:container_log_t:s0

Comment 8 Peter Portante 2017-08-22 17:30:17 UTC

Isn't this a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1477787?

The proper fix is in comment 10 (https://bugzilla.redhat.com/show_bug.cgi?id=1477787#c10).

Comment 9 Vivek Goyal 2017-08-22 17:50:08 UTC

Yes, this seems duplicate of 1477787.

Comment 10 Vivek Goyal 2017-08-22 17:53:43 UTC

Instead of removing "/var/lib/docker/containers", change it to "/var/lib/docker/containers/*" and make sure docker has been upgraded to version 
docker-2:1.12.6-52.gitc4618fb. And should resolve the issue.

Comment 11 Jeff Cantrill 2017-08-23 16:41:15 UTC

@Xia can this issue be closed as any fix appears to be unrelated to logging

Comment 12 Vivek Goyal 2017-08-23 16:57:11 UTC

@jeff, If we close this, how would we test that updated docker has fixed the issue. I mean docker QE will only verify regression testing but I am assuming that fluentd QE will have to take this bug and make sure fluentd does not have issues with new docker.

Comment 13 Vivek Goyal 2017-08-23 16:58:03 UTC

IMO, we should keep this bug around just to make sure fluentd is tested with new docker and make sure issue actually got fixed.

Comment 14 Rich Megginson 2017-08-23 18:30:53 UTC

We are in the process of adding testing logging with the json-file log driver to the test matrix.  Once we fix this bug, we'll be able to make that job gating.

Comment 15 Xia Zhao 2017-08-24 03:03:39 UTC

(In reply to Jeff Cantrill from comment #11)
> @Xia can this issue be closed as any fix appears to be unrelated to logging

@Jeff, yes, agree with @vgoyal that I would prefer to leave this bz open as a tracker to the docker bug fix in fluentd side. But consider that fact that this is not a logging bug, please also feel free to let me know if any inconvenience this may cause. I can close then. Thanks!

Comment 16 Jeff Cantrill 2017-08-25 19:09:50 UTC

Moving to ON_QA to close issue once validated by the dependent fix

Comment 17 Xia Zhao 2017-08-31 09:58:01 UTC

It's fixed, verified with docker Package version: docker-1.12.6-55.gitc4618fb.el7.x86_64, fluentd is able to collect logs in json-file log driver now, without any work around:

# oc exec logging-es-data-master-ponf3y0s-1-fbb0g -- curl -s -k --cert  /etc/elasticsearch/secret/admin-cert --key  /etc/elasticsearch/secret/admin-key https://localhost:9200/_cat/indices
green open .searchguard.logging-es-data-master-ponf3y0s                         1 0      5 0  27.6kb  27.6kb 
green open .operations.2017.08.31                                               1 0 158088 0  39.8mb  39.8mb 
green open .kibana                                                              1 0      1 0   3.1kb   3.1kb 
green open project.install-test.69c16b8d-8e29-11e7-9e8a-fa163ee02e8a.2017.08.31 1 0    294 0 172.4kb 172.4kb 
green open .kibana.f7724d98466ed7391e970202dc54a6460046aadb                     1 0      5 0  49.1kb  49.1kb 
green open project.logging.20d38cd2-8e29-11e7-9e8a-fa163ee02e8a.2017.08.31      1 0    333 0 180.2kb 180.2kb


Test env:
# docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-55.gitc4618fb.el7.x86_64
 Go version:      go1.8.3
 Git commit:      c4618fb/1.12.6
 Built:           Thu Aug 24 14:48:49 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-55.gitc4618fb.el7.x86_64
 Go version:      go1.8.3
 Git commit:      c4618fb/1.12.6
 Built:           Thu Aug 24 14:48:49 2017
 OS/Arch:         linux/amd64


# openshift version
openshift v3.6.173.0.21
kubernetes v1.6.1+5115d708d7
etcd 3.2.1

Set to verified.

Comment 18 Xia Zhao 2017-08-31 09:59:50 UTC

The warnings in fluentd log is also fixed:

# oc logs logging-fluentd-7t4j5
umounts of dead containers will fail. Ignoring...
umount: /var/lib/docker/containers/0cdf88dd3c65743f1130fb69ea4d9b53250d0ef88920cb406bb0711050a42f55/shm: not mounted
umount: /var/lib/docker/containers/0cf4eb4583cdc21ead5cb8186fe4586c85915cc887dd1e58ea4fe6c8bab75001/shm: not mounted
umount: /var/lib/docker/containers/38f86f90cd630e2c0d02657530545308658a01d9488bc30bc3f0a70714db315d/shm: not mounted
umount: /var/lib/docker/containers/3d4cfbba460e2d6c55cce06fc23a422c2f8a8ea2a9fbf979892af02660cb015b/shm: not mounted
umount: /var/lib/docker/containers/42081de051c6921458207d760c3f53334644078935daaa3ed05fff3adbf05105/shm: not mounted
umount: /var/lib/docker/containers/5877fb57b63206147b85b12075e04828cd3657fe2d0a8911594c028c10d9515d/shm: not mounted
umount: /var/lib/docker/containers/664287b67bed7e6d46b29d704d29be4f119040a8fe88366afcc35527bdb2107e/shm: not mounted
umount: /var/lib/docker/containers/6ddf714c20f4873631f2504ae6a0876c3e951d6212680b6fead86bd53ecae864/shm: not mounted
umount: /var/lib/docker/containers/72bd3d0fdc6a00398c272e1f9111c4024adc57dd9dcedb6a94314d619e6b7c5c/shm: not mounted
umount: /var/lib/docker/containers/829cc61bd12acb3450d104e80d1d653e4d0ae4d106842d07727f4f0105ec330f/shm: not mounted
umount: /var/lib/docker/containers/82f0ef5d2349ef5b0aed7dc1d735ad4132dac07db9461035ecdd911ce102091f/shm: not mounted
umount: /var/lib/docker/containers/97c05b95f3e937247fead90160e5c44a534283d60cfb8860ddce7b7a8ba5605a/shm: not mounted
umount: /var/lib/docker/containers/c5ecc41d47743e46b668ea4de2833f7f9c662a61cf424a43a98826f77fa76c5c/shm: not mounted
umount: /var/lib/docker/containers/c7313f53035ca771c8262fc2efe5764ad91ca144cc19b2047b22307c1d4eba73/shm: not mounted
umount: /var/lib/docker/containers/e0396a9aa7aaaa01b0b43f6b880aae5d74476a1805ff3c586bb48fc662555b41/shm: not mounted
umount: /var/lib/docker/containers/e2952617b905125a66cd119a187e16452d60f0afc883d57aa99558ed69d8e83d/shm: not mounted
umount: /var/lib/docker/containers/f55a2657c824855e897292d62af9de11afe7ef1fd8f26b137357013ea872df30/shm: not mounted
umount: /var/lib/docker/containers/fe1a04df867b750d26383f33b8501c68e0d2f9ddffb92450fa7b1e4c83f12446/shm: not mounted
2017-08-31 05:43:30 -0400 [info]: reading config file path="/etc/fluent/fluent.conf"
2017-08-31 05:43:33 -0400 [warn]: 'exclude1' parameter is deprecated: Use <exclude> section

Comment 19 Peter Portante 2017-08-31 10:45:17 UTC

Can you open a low-priority bug for this warning:

2017-08-31 05:43:33 -0400 [warn]: 'exclude1' parameter is deprecated: Use
<exclude> section

Thanks!

Comment 20 Noriko Hosoi 2017-08-31 17:48:08 UTC

(In reply to Peter Portante from comment #19)
> Can you open a low-priority bug for this warning:
> 
> 2017-08-31 05:43:33 -0400 [warn]: 'exclude1' parameter is deprecated: Use
> <exclude> section
> 
> Thanks!

Hi Peter, I happen to have a patch for the warning.  Is it ok to submit a pr from this diff?
https://github.com/openshift/origin-aggregated-logging/compare/master...nhosoi:exclude1

Comment 21 Peter Portante 2017-09-01 02:26:21 UTC

Yes, thanks for taking care of this, Noriko!

Comment 22 Noriko Hosoi 2017-09-01 05:12:15 UTC

(In reply to Peter Portante from comment #21)
> Yes, thanks for taking care of this, Noriko!

Thank you, Peter!  Here's the pr...
https://github.com/openshift/origin-aggregated-logging/pull/629

Comment 24 errata-xmlrpc 2017-10-25 13:04:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3049

Note You need to log in before you can comment on or make changes to this bug.