Red Hat Bugzilla – Bug 1517875
RFE: Allow regular expressions in curator settings
Last modified: 2018-05-02 07:54:10 EDT
Description of problem: I would like to delete some indices according to the regular expressions I provide, so that I am not forced to enter individual project names into the configs. Right now I am creating a setting yaml with a one-liner and add it to the configmap, manually. But whenever new projects are added to the cluster I will have to maintain the list. Version-Release number of selected component (if applicable): 3.5.1 How reproducible: Steps to Reproduce: 1. Install Cluster 2. Deploy logging with the provided playbook 3. Add the following to the configmap logging-curator .+nonprod.+: delete: days: 5 In order to delete the indices that belong to *nonprod projects. Actual results: Regex characters are escaped, as a result the indices are not deleted.
ovirt/rhv would like this feature as well
Bahaddin, we're currently moving to curator 5.2, which allows regexes in its configuration. See https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/filtertype_pattern.html#_regex Let me know if that helps you. Thanks
(In reply to Josef Karasek from comment #2) > Bahaddin, we're currently moving to curator 5.2, which allows regexes in its > configuration. See > https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/ > filtertype_pattern.html#_regex > > Let me know if that helps you. Thanks Josef, is there anything we can do in current version of curator?
(In reply to Josef Karasek from comment #2) > Bahaddin, we're currently moving to curator 5.2, which allows regexes in its > configuration. See > https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/ > filtertype_pattern.html#_regex > > Let me know if that helps you. Thanks Hi Josef, which OCP version will be using the curator 5.2? Thanks, Bahaddin
We're hoping to land this upgrade in early 2018. I understand that this might not satisfied your needs. I'll look into extending the current config file format to accept regular expressions as well.
https://github.com/openshift/origin-aggregated-logging/pull/862
The latest image logging-curator/images/v3.9.0-0.53.0.0 won't include the fix. Waiting the new image.
@josef, The curator report error, Could you Check if the configmap are correct? [root@ip-172-18-5-104 bin]# oc get configmap logging-curator -o yaml apiVersion: v1 data: config.yaml: | myapp-dev: delete: days: 1 myapp-qe: delete: weeks: 1 .operations: delete: weeks: 2 .defaults: delete: days: 31 runhour: 0 runminute: 0 timezone: America/New_York .regex: - pattern: '^project\..+\-dev\..*$' delete: days: 15 - pattern: '^project\..+\-test\..*$' delete: days: 30 kind: ConfigMap [root@ip-172-18-5-104 bin]# oc logs logging-curator-1-6sptm The project name must match this regex: [^[a-z0-9]([-a-z0-9]*[a-z0-9])?$] This does not match: [.regex]
The test fails because none of the latest builds have the latest code. I'm looking into it. The indentation of the config should be: config.yaml: | myapp-dev: delete: days: 1 myapp-qe: delete: weeks: 1 .operations: delete: weeks: 2 .defaults: delete: days: 31 runhour: 0 runminute: 0 timezone: America/New_York .regex: - pattern: '^project\..+\-dev\..*$' delete: days: 15 - pattern: '^project\..+\-test\..*$' delete: days: 30 You can find more info here https://github.com/openshift/origin-aggregated-logging/blob/release-3.9/curator/README.md
Moving back to RFE as this was a change enhancement. Updated dist-git repos to include latest files due to bad merge. Requires new build to validate.
The curator works as expected with openshift3/logging-curator/images/v3.9.11-1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489