Bug 1459384

Summary: [RFE] View application logs resides under /data directory in pod on OSD
Product: OpenShift Container Platform Reporter: Kenjiro Nakayama <knakayam>
Component: RFEAssignee: Jeff Cantrill <jcantril>
Status: CLOSED DEFERRED QA Contact: Xiaoli Tian <xtian>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5.1CC: aos-bugs, erich, jokerman, mbarrett, mmccomas, pweil
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-12 13:54:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kenjiro Nakayama 2017-06-07 02:24:48 UTC
1. Proposed title of this feature request
---
- View application logs resides under /data directory in pod on OSD.

3. What is the nature and description of the request?
---
- Customer wants to view the logs resides under /data directory in pods.

4. Why does the customer need this? (List the business requirements here)
---
- Their application output the logs /data directory as well, and it is useful if it could be visible on kibana dashboard.

5. How would the customer like to achieve this? (List the functional requirements here)
---
- Hopefuly, they want this feature on OSD.

7. Is there already an existing RFE upstream or in Red Hat Bugzilla?
---
- No. And here is the internal discussion ML threads - http://post-office.corp.redhat.com/archives/sbr-shift-list/2017-June/msg00031.html

10. List any affected packages or components.
---
- OpenShift Logging (kibana & fluentd)

Comment 2 Paul Weil 2017-08-08 18:45:32 UTC
Logging to storage within the container is not something that is accessible to Kibana via the log driver.  If it was, it would need to be configured at a system level for individual pods which makes it hard to maintain for system administrators.

If the user would like to see their pod logs via aggregated logging they will need to ensure that they are logging to stdout/stderr which is normal for containerized applications.  Otherwise they are responsible for retrieving, displaying, and rotation of the files they are creating

Comment 3 Kenjiro Nakayama 2017-08-08 23:43:43 UTC
Paul, I filed this bug ticket because Logging team (Jeff) asked me to open a ticket for this request. Logging team also agreed on c#2?

Comment 6 Paul Weil 2017-08-09 14:05:26 UTC
I spoke with our logging team about this.  Right now the only way logs are gathered (as noted in the email) is by ensuring the logging driver is able to pick up (ie it is showing in docker logs).  It is possible to ship logs via more complicated methods by configuration but as part of the system it requires applications to know about the infrastructure as well as a way for admins to control that knowledge.

The info Jeff discusses in the email is basically to use a side car container that can ensure the non-stdout/stderr logs are available - think of a simple side car that can tail the log while the main app is dealing with rotation and clean up.  This would make it available to the logging driver.

However, there is also some talk upstream about providing logging specific resources: https://github.com/kubernetes/kubernetes/issues/24677  

Based on that I think we have two things.  First we need some good documentation on how this can be accomplished today.  Second we can make a card for tracking the upstream direction on this effort (https://trello.com/c/5GUvDaQU).

Reopening.

Comment 7 Eric Rich 2018-03-12 13:54:36 UTC
This bug has been identified as a dated (created more than 3 months ago) bug. 
This bug has been triaged (has a trello card linked to it), or reviewed by Engineering/PM and has been put into the product backlog, 
however this bug has not been slated for a currently planned release (3.9, 3.10 or 3.11), which cover our releases for the rest of the calendar year. 

As a result of this bugs age, state on the current roadmap and PM Score (being below 70), this bug is being Closed - Differed, 
as it is currently not part of the products immediate priorities.

Please see: https://docs.google.com/document/d/1zdqF4rB3ea8GmVIZ7qWCVYUaQ7-EexUrQEF0MTwdDkw/edit for more details.