Bug 1304791

Summary:	[RFE] Send aggregate container logs to an external source as well as ElasticSearch
Product:	OpenShift Container Platform	Reporter:	Eric Jones <erjones>
Component:	RFE	Assignee:	Rich Megginson <rmeggins>
Status:	CLOSED ERRATA	QA Contact:	Johnny Liu <jialiu>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.3.0	CC:	aos-bugs, bfallonf, drettori, erich, eyenerol, jcantril, jokerman, kurktchiev, misalunk, mmccomas, rmeggins, steven, stwalter, tdawson
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1337331 (view as bug list)		Environment:
Last Closed:	2016-09-27 09:35:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Eric Jones 2016-02-04 15:54:53 UTC

- What is the nature and description of the request? 
As an admin I need the ability to send the aggregate container logs to an external source so that they can be archived.

- Why does the customer need this? (List the business requirements here) 
They wish to be able to store the aggregated container logs externally and archive them for future reference.

- How would the customer like to achieve this? (List the functional requirements here) 
Allow for an option to be passed into the Fluentd creation that opens a port that can be used externally, and/or provide the certs necessary to have Fluentd pass the logs to another source in addition to the ElasticSearch pod.

- For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.
Deploy the aggregate container logs and send a copy of the logs to an external source.

- Is there already an existing RFE upstream or in Red Hat Bugzilla?
Not that I could find.

- List any affected packages or components.
Logging
Fluentd

Comment 8 Rich Megginson 2016-04-18 15:28:16 UTC

Requesting RH AOS 3.2.0 acks

Comment 25 Boris Kurktchiev 2016-08-09 12:57:33 UTC

So there seems to be a lot of info on this thats contradicting. Here you have it slated for 3.3 release, but i believe the trello card with this says 3.4. Which one is true as that dictates things on when/how we can deployer v3 to our customers?

Comment 26 Rich Megginson 2016-08-09 14:12:05 UTC

(In reply to Boris Kurktchiev from comment #25)
> So there seems to be a lot of info on this thats contradicting. Here you
> have it slated for 3.3 release, but i believe the trello card with this says
> 3.4. Which one is true as that dictates things on when/how we can deployer
> v3 to our customers?

This feature is in 3.3.  Which trello card are you looking at?

Comment 27 Boris Kurktchiev 2016-08-09 15:32:08 UTC

Seemingly I can no longer find the one I had a link to before... but if its slated for 3.3 thats good news!

Comment 28 Jeff Cantrill 2016-08-09 16:03:07 UTC

(In reply to Boris Kurktchiev from comment #27)
> Seemingly I can no longer find the one I had a link to before... but if its
> slated for 3.3 thats good news!

We should clarify, the linked card in this BZ https://trello.com/c/CA20axj2 is to copy logs to an ES instance outside an openshift cluster.  There additionally is https://trello.com/c/j9BAcanp which addresses sending logs to aggregators (e.g. splunk) other then ES which is targted for 3.4

Comment 29 Steven Walter 2016-09-13 14:56:54 UTC

Why are there docs at 
https://docs.openshift.com/enterprise/3.2/install_config/aggregate_logging.html#sending-logs-to-an-external-elasticsearch-instance
and
https://docs.openshift.com/enterprise/3.1/install_config/aggregate_logging.html#external-elasticsearch-instance-with-fluentd

If this feature is not available until 3.3? Are these docs to *replace* the destination, whereas the RFE is to add an *additional* destination? Are the docs a mistake?

Comment 30 Rich Megginson 2016-09-13 15:03:08 UTC

(In reply to Steven Walter from comment #29)
> Why are there docs at 
> https://docs.openshift.com/enterprise/3.2/install_config/aggregate_logging.
> html#sending-logs-to-an-external-elasticsearch-instance
> and
> https://docs.openshift.com/enterprise/3.1/install_config/aggregate_logging.
> html#external-elasticsearch-instance-with-fluentd
> 
> If this feature is not available until 3.3? Are these docs to *replace* the
> destination, whereas the RFE is to add an *additional* destination?

Yes.

> Are the docs a mistake?

No.

For contrast, these are the instructions for using the ES Copy feature: https://github.com/openshift/origin-aggregated-logging/blob/master/deployer/README.md#have-fluentd-send-logs-to-another-elasticsearch

Comment 31 Steven Walter 2016-09-13 15:05:27 UTC

Cool, thanks Rich!

Comment 33 errata-xmlrpc 2016-09-27 09:35:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933