Hide Forgot
From the RFE template: 3. What is the nature and description of the request? The customer wants to use external elasticsearch (especially AWS managed elasticsearch service) instead of embedded one. Red Hat support shipping logs to external elasticsearch with fluentd template, but curator only support embedded elasticsearch. It's reasonable to support external elasticsearch. 4. Why does the customer need this? (List the business requirements here) Some customers want to AWS elasticsearch or other external elasticsearches for some business reason. AWS elasticsearch is a managed service, so it has a merit to use. 5. How would the customer like to achieve this? (List the functional requirements here) Same as 3. 6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented. Example scenario: Create AWS elasticseach instance and ship logs from fluentd to elasticsearch. Then the curator can manage this external elasticssearch as the same as the embedded one. 10. List any affected packages or components. Logging template. Especially, logging-curator-template and logging-curator image.
I forgot the customer's comment 6. I guess the way to validate it would be to make sure that AWS ES indices are deleted/rotated as per the curator settings. Basically the same way functionality curator currently implements with the embedded ES instance.
What exactly is the enhancement for this request? It is currently possible to deploy 3.3 curator image to manage an external ES cluster, assuming the cluster is visible to openshift and has access to the certs: https://github.com/openshift/origin-aggregated-logging/blob/v1.3.1/curator/run_cron.py#L80 and configuration file: https://github.com/openshift/origin-aggregated-logging/blob/v1.3.1/curator/run_cron.py#L53. Alternatively, I expect with minimal modifications, this script could be added as a cron job to the host running the ES cluster.