Bug 1643341 - [RFE] [3.10] Fluentd pod should log its logs to /var/log/fluentd/fluentd.log by default
Summary: [RFE] [3.10] Fluentd pod should log its logs to /var/log/fluentd/fluentd.log ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.10.z
Assignee: Noriko Hosoi
QA Contact: Qiaoling Tang
URL:
Whiteboard:
Depends On: 1643340 1649997 1650005
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-10-26 03:40 UTC by Qiaoling Tang
Modified: 2018-12-13 17:09 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Running "oc logs $fluentd_pod" suggests to run "oc exec <pod_name> /opt/app-root/src/utils/logs" which includes the non existing utility logs path. Consequence: "oc exec <pod_name> /opt/app-root/src/utils/logs" fails with "no such file or directory". Fix: Fixed the suggested command line to "oc exec <pod_name> -- logs" since the utility "logs" is now in the PATH and no need to specify the full path. Result: "oc logs $fluentd_pod" suggests the correct command line to show the fluentd logs.
Clone Of: 1643340
Environment:
Last Closed: 2018-12-13 17:09:08 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:3750 None None None 2018-12-13 17:09:16 UTC

Description Qiaoling Tang 2018-10-26 03:40:57 UTC
+++ This bug was initially created as a clone of Bug #1643340 +++

Description of problem:
Couldn't find fluentd pod log file.

After a deployment of logging EFK stack, execute "oc logs $fluentd-pod", the output are in the below:
=============================
Fluentd logs have been redirected to: /var/log/fluentd/fluentd.log
If you want to print out the logs, use command:
oc exec <pod_name> /opt/app-root/src/utils/logs
=============================
umounts of dead containers will fail. Ignoring...
umount: /var/lib/docker/containers/2ed37f4b5460c50224003b667cc8a80772290c296df1262f331cf8022aafc27a/shm: not mounted
umount: /var/lib/docker/containers/3bebfa9be927ad765eacc17dbd9a240c3e5034d15d4d7c7ac8d391de8851450c/shm: not mounted
umount: /var/lib/docker/containers/4391779ea9805a582a3b28cc9003c45f48777ef2a9303a0f91cb5c7097d9c007/shm: not mounted
umount: /var/lib/docker/containers/63ca3d8ab9cc7518e589777e1d981a1615871439b6cebd3fe20689679bd1a61d/shm: not mounted
umount: /var/lib/docker/containers/6ce5fb53d7bdd1686d9c70c81397a836b0505a8949a30f7efb4b23abd5920a1f/shm: not mounted
umount: /var/lib/docker/containers/829088d8f33a5454badcbae52adedb3b47a9ff77995f6f2c09fc69f54a14353d/shm: not mounted
umount: /var/lib/docker/containers/ab0f5b8ad0758c2778319d1327000b5ca6b0ba736a5c7c694457d9bbc5d5da59/shm: not mounted
umount: /var/lib/docker/containers/ce952648d783c077db72486f2b56020b990e6aa55ffa6858e6b2b61e6a1fae24/shm: not mounted
umount: /var/lib/docker/containers/d1dadd88a755d5163b8f194ddc34300b71d726aa27ff3e6724b5389bd5ec4010/shm: not mounted
[DEPRECATION WARNING]: 'merge_json_log' for the fluent-plugin-kubernetes_metadata_filter will no longer be supported in future releases
2018-10-25 22:48:33 -0400 [info]: reading config file path="/etc/fluent/fluent.conf"
2018-10-25 22:48:44 -0400 [warn]: 'block' action stops input process until the buffer full is resolved. Check your pipeline this action is fit or not
2018-10-25 22:48:44 -0400 [warn]: 'block' action stops input process until the buffer full is resolved. Check your pipeline this action is fit or not
2018-10-25 22:49:22 -0400 [warn]: temporarily failed to flush the buffer. next_retry=2018-10-25 22:48:47 -0400 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"logging-es\", :port=>9200, :scheme=>\"https\", :user=>\"fluentd\", :password=>\"obfuscated\"})!" plugin_id="elasticsearch-apps"
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluent-plugin-elasticsearch-1.17.2/lib/fluent/plugin/out_elasticsearch.rb:228:in `client'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluent-plugin-elasticsearch-1.17.2/lib/fluent/plugin/out_elasticsearch.rb:477:in `rescue in send_bulk'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluent-plugin-elasticsearch-1.17.2/lib/fluent/plugin/out_elasticsearch.rb:464:in `send_bulk'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluent-plugin-elasticsearch-1.17.2/lib/fluent/plugin/out_elasticsearch.rb:382:in `write_objects'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/output.rb:490:in `write'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/buffer.rb:354:in `write_chunk'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/buffer.rb:333:in `pop'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/output.rb:342:in `try_flush'
  2018-10-25 22:49:22 -0400 [warn]: /usr/share/gems/gems/fluentd-0.12.43/lib/fluent/output.rb:149:in `run'
2018-10-25 22:49:58 -0400 [warn]: temporarily failed to flush the buffer. next_retry=2018-10-25 22:48:49 -0400 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"logging-es\", :port=>9200, :scheme=>\"https\", :user=>\"fluentd\", :password=>\"obfuscated\"})!" plugin_id="elasticsearch-apps"
  2018-10-25 22:49:58 -0400 [warn]: suppressed same stacktrace
2018-10-25 22:50:08 -0400 [warn]: retry succeeded. plugin_id="elasticsearch-apps"

rsh into a fluentd pod, there is no fluentd.log file in /var/log/fluentd directory, and couldn't find /opt/app-root/src/utils/logs either.

No output shows after run "oc exec $fluentd-pod -- logs".

Version-Release number of selected component (if applicable):
logging-fluentd-v3.10.64-1

How reproducible:
Always

Steps to Reproduce:
1.Deploy logging 

Actual results:
Couldn't find fluentd pod log file.

Expected results:
Fluentd should log its log to a file by default.

Additional info:

Comment 5 Qiaoling Tang 2018-11-29 07:29:38 UTC
Verified in logging-fluentd-v3.10.79-1.

Comment 7 errata-xmlrpc 2018-12-13 17:09:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3750


Note You need to log in before you can comment on or make changes to this bug.